-- Logs begin at Fri 2022-09-09 12:04:45 UTC, end at Fri 2022-09-09 13:16:50 UTC. -- Sep 09 12:35:54.356389 dv-cinder-813229-14-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 12:35:54.365972 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 12:35:54.367540 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 12:35:54 2022] *** Sep 09 12:35:54.367540 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 12:35:54.367540 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 12:35:54.367540 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: nodename: dv-cinder-813229-14-iscsi Sep 09 12:35:54.367540 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: machine: x86_64 Sep 09 12:35:54.367540 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: clock source: unix Sep 09 12:35:54.367540 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: pcre jit disabled Sep 09 12:35:54.367540 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: detected number of CPU cores: 4 Sep 09 12:35:54.367540 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: current working directory: / Sep 09 12:35:54.367540 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: detected binary path: /usr/bin/uwsgi-core Sep 09 12:35:54.367540 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: your processes number limit is 47680 Sep 09 12:35:54.367540 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: your memory page size is 4096 bytes Sep 09 12:35:54.367540 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: detected max file descriptor number: 2048 Sep 09 12:35:54.368249 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: lock engine: pthread robust mutexes Sep 09 12:35:54.368249 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: thunder lock: enabled Sep 09 12:35:54.368249 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 12:35:54.368249 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 12:35:54.406426 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: Python main interpreter initialized at 0x55dbbae929c0 Sep 09 12:35:54.406426 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: python threads support enabled Sep 09 12:35:54.406426 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: your server socket listen backlog is limited to 100 connections Sep 09 12:35:54.406426 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: your mercy for graceful operations on workers is 90 seconds Sep 09 12:35:54.406790 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 12:35:54.406790 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: *** Operational MODE: preforking *** Sep 09 12:35:54.407059 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: *** uWSGI is running in multiple interpreter mode *** Sep 09 12:35:54.407059 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: spawned uWSGI master process (pid: 88026) Sep 09 12:35:54.407272 dv-cinder-813229-14-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 12:35:54.408286 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: spawned uWSGI worker 1 (pid: 88041, cores: 1) Sep 09 12:35:54.408286 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: spawned uWSGI worker 2 (pid: 88042, cores: 1) Sep 09 12:35:54.408286 dv-cinder-813229-14-iscsi devstack@keystone.service[88026]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 12:35:58.719481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 12:35:58.719481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: from cryptography.utils import int_from_bytes Sep 09 12:35:58.720280 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 12:35:58.720280 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: from cryptography.utils import int_from_bytes Sep 09 12:35:58.725616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 12:35:58.725616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: from cryptography.utils import int_from_bytes Sep 09 12:35:58.726672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 12:35:58.726672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: from cryptography.utils import int_from_bytes Sep 09 12:35:58.770711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 12:35:58.770897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 12:35:58.771142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] command line args: [] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 12:35:58.771419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 12:35:58.771602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ================================================================================ {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 12:35:58.771859 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] admin_token = **** {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.772108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] config_dir = [] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.772299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.772485 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] config_source = [] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.772670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] debug = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.772873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.773715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 12:35:58.773796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.773902 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 12:35:58.774046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.774148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] command line args: [] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 12:35:58.774310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.774417 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 12:35:58.774535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ================================================================================ {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 12:35:58.774653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.774754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] admin_token = **** {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.774863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] list_limit = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.774987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] log_config_append = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.775095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] config_dir = [] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.775194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.775294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.775393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] log_dir = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.775490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] config_source = [] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.775591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] log_file = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.775689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] debug = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.775689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.775872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.775954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.776076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.776155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.776267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.776348 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.776425 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.776542 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.776621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.776740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] list_limit = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.776814 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.777235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] log_config_append = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.777322 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.777446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.777446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] log_dir = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.777632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.777717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] log_file = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.777794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.777907 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.777980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.778098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.778177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.778255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.778325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.778396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.778466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.778592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.778668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.778794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.778882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.778959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.779108 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.779340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.779427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.779571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.779653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] publish_errors = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.779802 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.779923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.780029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.780109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.780215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.780297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.780440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.780528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.780678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.780754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.780914 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] standard_threads = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.781147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.781223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.781393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.781491 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] publish_errors = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.781615 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.781738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.781849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] use_journal = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.781950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.782057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] use_json = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.782168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.782284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] use_stderr = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.782405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.782536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] use_syslog = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.782649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.782769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.782769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.782977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] standard_threads = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.783094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.783212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.783323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.783432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.783432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.783643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.783787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] use_journal = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.783893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.783985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] use_json = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.783985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] use_stderr = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.784162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.784254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] use_syslog = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.784424 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.784519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 12:35:58.784618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.784716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.784809 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.784900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.785006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.785106 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.785199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.785290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.785387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.785478 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.785576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.785670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.785766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.785858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.786037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.786147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.786246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.786354 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.786445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.788258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.788365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.788490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.788630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.789224 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.789305 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.789376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.789445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.789525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.789596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.789670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.789670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.789802 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.789872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.789948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.790019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.790094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.790169 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.790243 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.790243 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.790381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.790451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.790569 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.790640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.790709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.790777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.790924 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.790997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.791066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.791134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.791202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.791270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.791348 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.791418 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.791492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.791561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.791629 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.791702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.791779 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.791854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.791924 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.792042 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.792156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.792268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.792369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.792469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.792568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.792712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.792819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.792940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.793057 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.793190 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.793312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.793394 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.793469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.793651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.793793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.793872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.793940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.794096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.794217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.794328 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.794410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.794590 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.794686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.794822 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.794907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.795051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.795132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.795285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.795361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.795516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.795601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.795743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.795835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.795976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.796065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.796205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.796284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.796438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.796523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.796668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.796747 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.796896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.796979 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.797128 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.797222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.797362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.797443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.797595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.797671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.797825 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.797922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.798056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.798142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.798288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.798369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.798535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.798619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.798781 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.798867 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.799008 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.799087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.799246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.799325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.799474 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.799556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.799837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.799961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.800043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.800189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.800271 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.800421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.800507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.800653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.800750 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.800917 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.800999 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.801150 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.801230 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.801385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.801463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.801611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.801686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.801842 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.801922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.802073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.802153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.802304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.802384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.802557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.802640 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.802791 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.802887 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.803025 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.803113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.803261 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.803341 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.803490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.803568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.803733 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.803828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.803964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.804046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.804240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.804320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.804540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.804631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.804844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.804923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.805086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.805166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.805325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.805413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.805672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.805756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.805837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.805916 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.806052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.806141 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.806284 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.806364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.806535 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.806619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.806771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.806864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.807002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.807332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.807491 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.807571 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.807720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.807810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.807949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.808029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.808175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.808256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.808406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.808495 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.808632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.808729 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.808869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.808949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] auth.external = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.809101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.809190 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.809365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.809486 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.809594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.809682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.809823 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.809911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] auth.password = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.810070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] auth.token = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.810158 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.810306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.810386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.810558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] auth.external = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.810641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.810792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.810885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.811030 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.811173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.811299 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.811380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.811526 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] auth.password = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.811616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.811753 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] auth.token = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.811834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.811989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.812076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] credential.caching = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.812221 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.812310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.812464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.812545 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.812691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.812772 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.812936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.813018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.813179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.813260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.813409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.813490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.813640 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] credential.caching = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.813739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.813885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.813965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.814121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.814210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.814364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.814444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.814636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.814719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.814879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.814959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.815111 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.815194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.815356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.815437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.815608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.815697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.815849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.815926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] federation.caching = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.816072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.816152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.816322 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.816403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.816606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.816729 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.816856 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.816940 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.817068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.817147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.817303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.817383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.817539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] federation.caching = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.817619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.817775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.817868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.818018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.818099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.818255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.818335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.818534 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.818619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.818747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.818830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.caching = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.818980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.819061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.819230 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.819406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.819490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.819564 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.819687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.819769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.819958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.820037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.820196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.820344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.820482 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.820563 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.caching = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.820676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.820810 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.820929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.821068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.821195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.821274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.821381 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.821502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.821596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.821745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.821826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.821972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.822073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.822206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.822320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.822450 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.822603 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.822719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.822853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.834171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.834286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.834367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.834440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.834559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.834635 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.834722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.834809 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.834884 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.834956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.835034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.835108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.835188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.835268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.835342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.835420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.835990 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836207 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836352 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836424 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836690 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.836983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.837058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.837130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.837201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.837273 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.837344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.837424 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.837498 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.837570 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.837642 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.837715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.837787 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.837866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.837941 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838683 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838767 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.838993 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.839066 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.839139 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.839219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.839295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.839367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.839440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.839512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.839584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.839665 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.839768 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.839848 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.839917 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.839990 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.840062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.840143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.840217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.840297 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.840371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.840443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.840515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.840595 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.840669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.840741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.840813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.840884 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.840956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841036 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841109 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841181 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841610 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841754 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.841980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.842052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.842123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.842195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.842267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.842267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.842406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.842494 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.842571 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.842643 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.842728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.842802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.842874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.842946 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843169 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843533 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.843974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.844044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.844117 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.844189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.844261 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.844340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.844413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.844556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.844632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.844705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.844777 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.844849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.844920 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845000 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] resource.caching = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845560 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845809 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.845965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846196 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] resource.caching = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846350 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846424 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846595 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846716 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.846975 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] role.caching = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] role.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847224 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847284 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847463 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] role.caching = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] role.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847767 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847886 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.847954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848194 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848322 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848444 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848698 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848921 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848993 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.848993 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.849133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.849206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.849278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.849350 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.849421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.849501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.849575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.849575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.849713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.849713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.849713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.849713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.849713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850571 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850643 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.850950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.851133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.851738 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.851850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.851932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.851994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.852066 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.852138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.852210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.852291 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.852366 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.852438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.852509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.852581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.852653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.852733 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] token.caching = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.852807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] token.caching = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.852878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.852950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.853022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.853093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.853173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.853247 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.853325 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.853402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.853474 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.853545 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.853625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.853699 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.853771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.853889 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.853973 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.854131 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.854222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.854366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.854448 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.854627 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.854710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.854861 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.854956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.855094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.855175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.855343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.855423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.855576 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.855665 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.855816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.855898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.856071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.856159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.856323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.connection = **** {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.856402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.856564 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.856658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.connection = **** {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.856808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.856890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.857042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.857123 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.857279 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.857368 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.857511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.857592 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.857744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.857824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.857975 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.858135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.858213 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.858286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.858435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.858552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.858709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.858804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.858945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.859025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.859174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.859253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.859414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.859502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.859651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.859730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.859886 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.859965 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.860111 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.860199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.860346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.860430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.860577 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.860658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.860731 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88041) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 12:35:58.860814 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 12:35:58.860966 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88042) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 12:35:58.897813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:58.898598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:58.898826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:58.899288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:58.899522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:58.899748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:58.907148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:58.908163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:58.935587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:58.935814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:58.938281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:58.938528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:58.993408 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:58.993796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.042084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.042311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.042786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.043025 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.205646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.205879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.206096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.206311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.206554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.206770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.209395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.209628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.209852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.210071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.210291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.210654 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.229788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.230037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.230142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.230225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.230348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.230436 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.230602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.230688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.230831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.230928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.231050 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.231134 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.231261 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.231472 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.242171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.242643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.254784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.254960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.255102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.255186 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.255339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.255475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.255555 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.255631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.255769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.255866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.255978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88041) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.256062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.256191 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.256406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88042) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 12:35:59.286709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.286913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 12:35:59.287136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.287256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.287339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 12:35:59.287455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 12:35:59.287674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.287759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.287876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 12:35:59.287959 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.288151 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.288233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.288414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.288497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.288777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.290770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.298165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.306028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.306028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.306028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.306310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.306310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.306310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.306310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.306630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.309247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.309530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.309813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.310096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.310380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.310672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.310941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.311097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.311200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.311470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.311745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.312017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.314049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.314338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.314655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.314943 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.315226 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.315488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.315759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.316035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.316307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.316581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.316851 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.334890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.336978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.337262 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.340891 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.341892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.342176 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.342356 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.342455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.342666 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.347317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.347607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.347898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.348737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.349025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.354344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.354661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.354816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.355113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.355402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.355689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.355970 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.356248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.356521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.356791 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.360426 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.360724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.361015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.361305 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.361584 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.361858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.362129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.362398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.372155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.372463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.376288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.377983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.378260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.379556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.379854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.382175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.384589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.387963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.389705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.390023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.391168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.391471 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.391757 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.392039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.392314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.392612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.394262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.398070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.401337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.401639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.401928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.402210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.402502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.402789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.408349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.408639 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.408906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.409172 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.416598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.418125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.419253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.419544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.419666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.419812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.419964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.420063 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.422741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.423029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.423310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.423594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.423987 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.427053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.428598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.430841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.431118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.434116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.434398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.434706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.434991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.435391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.436181 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.436465 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.436742 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.437017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.437391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.437672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.437959 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.446505 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.446795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.447133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.447415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.447795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.448078 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.448365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.453355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.456258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.459133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.460622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.462117 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.462399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.462816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.463935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.466865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.469094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.469739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.470548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.471319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.472905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.473190 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.473391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.473585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.473785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.474071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.474387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.480759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.482217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.482720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.483011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.483282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.485165 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.485562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.485855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.486140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.490109 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88041) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.490387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.490686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.490960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.491232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.492738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.492738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.492738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.493008 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.493294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.493575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.493845 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.494114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.494392 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88041) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.494766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.495074 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.495373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.502276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88042) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 12:35:59.502598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.502885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.503164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.503440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.503716 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.503991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.504268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.504537 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.504857 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.505100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.505377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.505650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.505911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88042) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 12:35:59.529198 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55dbbae929c0 pid: 88041 (default app) Sep 09 12:35:59.532469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d454185-14f4-40d6-b1e0-b0b7e5990755 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:35:59.532755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d454185-14f4-40d6-b1e0-b0b7e5990755 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:35:59.533024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d454185-14f4-40d6-b1e0-b0b7e5990755 None None] PATH_INFO: `/v3` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:35:59.537126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Fri Sep 9 12:35:59 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:35:59.541171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55dbbae929c0 pid: 88042 (default app) Sep 09 12:36:07.244066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0fb89-a7ea-4310-8444-9c898da022bd None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:07.244576 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0fb89-a7ea-4310-8444-9c898da022bd None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:07.244656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0fb89-a7ea-4310-8444-9c898da022bd None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:07.245903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 2/2] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:07.253299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957ea30-1c7a-4727-b1de-7900e02cf026 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:07.253730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957ea30-1c7a-4727-b1de-7900e02cf026 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:07.253846 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957ea30-1c7a-4727-b1de-7900e02cf026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:07.324860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: INFO dbcounter [None req-3957ea30-1c7a-4727-b1de-7900e02cf026 None None] Registered counter for database keystone Sep 09 12:36:07.344185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG oslo_db.sqlalchemy.engines [None req-3957ea30-1c7a-4727-b1de-7900e02cf026 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88042) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 12:36:07.350258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writer thread running {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 12:36:07.527888 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG passlib.handlers.bcrypt [None req-3957ea30-1c7a-4727-b1de-7900e02cf026 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88042) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 12:36:07.528276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG passlib.handlers.bcrypt [None req-3957ea30-1c7a-4727-b1de-7900e02cf026 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88042) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 12:36:07.585438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-3957ea30-1c7a-4727-b1de-7900e02cf026 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:07.587663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3957ea30-1c7a-4727-b1de-7900e02cf026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:07.714752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1/3] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 12:36:08.234001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0fb89-a7ea-4310-8444-9c898da022bd None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:08.317408 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: INFO dbcounter [None req-bff0fb89-a7ea-4310-8444-9c898da022bd None None] Registered counter for database keystone Sep 09 12:36:08.344959 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG oslo_db.sqlalchemy.engines [None req-bff0fb89-a7ea-4310-8444-9c898da022bd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88041) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 12:36:08.352038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writer thread running {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 12:36:08.474469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0fb89-a7ea-4310-8444-9c898da022bd None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:08.479604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71346bfb-4c04-4d59-b057-f7574c239036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:08.480227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71346bfb-4c04-4d59-b057-f7574c239036 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:08.480503 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71346bfb-4c04-4d59-b057-f7574c239036 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:08.481036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71346bfb-4c04-4d59-b057-f7574c239036 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:08.489901 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71346bfb-4c04-4d59-b057-f7574c239036 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:08.550059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: WARNING keystone.common.rbac_enforcer.enforcer [None req-71346bfb-4c04-4d59-b057-f7574c239036 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 12:36:08.551281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71346bfb-4c04-4d59-b057-f7574c239036 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:08.563863 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 3/4] 10.222.0.46 () {58 vars in 1324 bytes} [Fri Sep 9 12:36:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 12:36:08.572417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957ea30-1c7a-4727-b1de-7900e02cf026 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:08.588560 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957ea30-1c7a-4727-b1de-7900e02cf026 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:08.593016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce570d33-750b-4c3b-82cc-bb93cd62b88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:08.594136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce570d33-750b-4c3b-82cc-bb93cd62b88e admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:08.594413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce570d33-750b-4c3b-82cc-bb93cd62b88e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:08.594712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce570d33-750b-4c3b-82cc-bb93cd62b88e admin admin] PATH_INFO: `/v3/projects/e314473e6b1442dfa9c0bcc6fc5f97c6` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:08.596145 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce570d33-750b-4c3b-82cc-bb93cd62b88e admin admin] RBAC: Authorizing `identity:get_project(project_id=e314473e6b1442dfa9c0bcc6fc5f97c6)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:08.653727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ce570d33-750b-4c3b-82cc-bb93cd62b88e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 12:36:08.654066 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce570d33-750b-4c3b-82cc-bb93cd62b88e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:08.659646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 2/5] 10.222.0.46 () {56 vars in 1207 bytes} [Fri Sep 9 12:36:08 2022] GET /identity/v3/projects/e314473e6b1442dfa9c0bcc6fc5f97c6 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:10.810336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c20cf-513f-4694-a9a6-b92891de6de9 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:10.810892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c20cf-513f-4694-a9a6-b92891de6de9 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:10.810892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c20cf-513f-4694-a9a6-b92891de6de9 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:10.814952 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 4/6] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:36:10.820399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bbe90-75fc-4965-a54c-8d75f78acc05 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:10.821659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bbe90-75fc-4965-a54c-8d75f78acc05 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:10.821659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bbe90-75fc-4965-a54c-8d75f78acc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:10.841466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-bc1bbe90-75fc-4965-a54c-8d75f78acc05 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:10.842990 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-bc1bbe90-75fc-4965-a54c-8d75f78acc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:10.850744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 3/7] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 12:36:11.127946 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c20cf-513f-4694-a9a6-b92891de6de9 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:11.141356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c20cf-513f-4694-a9a6-b92891de6de9 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:11.145663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56acff-dd2c-4741-8f1d-9e0f34cfdaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:11.146267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56acff-dd2c-4741-8f1d-9e0f34cfdaf2 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:11.146580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56acff-dd2c-4741-8f1d-9e0f34cfdaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:11.146853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56acff-dd2c-4741-8f1d-9e0f34cfdaf2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:11.155339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56acff-dd2c-4741-8f1d-9e0f34cfdaf2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:11.157086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56acff-dd2c-4741-8f1d-9e0f34cfdaf2 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:11.166309 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 5/8] 10.222.0.46 () {58 vars in 1324 bytes} [Fri Sep 9 12:36:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:36:11.173168 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bbe90-75fc-4965-a54c-8d75f78acc05 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:11.183560 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bbe90-75fc-4965-a54c-8d75f78acc05 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:11.187852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2f8c8-62a0-4429-a06a-f277d9b821ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:11.188471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2f8c8-62a0-4429-a06a-f277d9b821ac admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:11.188750 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2f8c8-62a0-4429-a06a-f277d9b821ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:11.189020 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2f8c8-62a0-4429-a06a-f277d9b821ac admin admin] PATH_INFO: `/v3/users/9947df19bb3b44318e674b5168544c18` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:11.190301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2f8c8-62a0-4429-a06a-f277d9b821ac admin admin] RBAC: Authorizing `identity:get_user(user_id=9947df19bb3b44318e674b5168544c18)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:11.192147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2f8c8-62a0-4429-a06a-f277d9b821ac admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:11.193646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 4/9] 10.222.0.46 () {56 vars in 1198 bytes} [Fri Sep 9 12:36:11 2022] GET /identity/v3/users/9947df19bb3b44318e674b5168544c18 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:13.427225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248c1bb-0448-4356-af85-e67af8691a3e None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:13.427798 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248c1bb-0448-4356-af85-e67af8691a3e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:13.427798 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248c1bb-0448-4356-af85-e67af8691a3e None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:13.429040 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 6/10] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:13.435053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ee138-2b45-404a-b5e8-c6903cda0348 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:13.435451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ee138-2b45-404a-b5e8-c6903cda0348 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:13.435609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ee138-2b45-404a-b5e8-c6903cda0348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:13.454268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-fb6ee138-2b45-404a-b5e8-c6903cda0348 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:13.455504 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-fb6ee138-2b45-404a-b5e8-c6903cda0348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:13.523692 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd6ef0-195d-41dc-9012-7b709b0503e5 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:13.524501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd6ef0-195d-41dc-9012-7b709b0503e5 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:13.524763 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd6ef0-195d-41dc-9012-7b709b0503e5 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:13.526094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 7/11] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:36:13.531373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba5f0e-9ced-44a4-b4b9-0c830f1665b4 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:13.531658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba5f0e-9ced-44a4-b4b9-0c830f1665b4 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:13.532182 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba5f0e-9ced-44a4-b4b9-0c830f1665b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:13.535481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 5/12] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:13.574119 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG passlib.handlers.bcrypt [None req-33ba5f0e-9ced-44a4-b4b9-0c830f1665b4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88041) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 12:36:13.574542 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG passlib.handlers.bcrypt [None req-33ba5f0e-9ced-44a4-b4b9-0c830f1665b4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88041) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 12:36:13.623320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-33ba5f0e-9ced-44a4-b4b9-0c830f1665b4 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:13.624863 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-33ba5f0e-9ced-44a4-b4b9-0c830f1665b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:13.722634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 8/13] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:13.812215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ee138-2b45-404a-b5e8-c6903cda0348 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:13.835922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:13.917383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ee138-2b45-404a-b5e8-c6903cda0348 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:13.951506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc0f57-a86d-4df7-88fa-234602746687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:13.952136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc0f57-a86d-4df7-88fa-234602746687 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:13.952415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc0f57-a86d-4df7-88fa-234602746687 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:13.952686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc0f57-a86d-4df7-88fa-234602746687 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:13.963559 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc0f57-a86d-4df7-88fa-234602746687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-6cfc0f57-a86d-4df7-88fa-234602746687 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:13.972233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:13.973313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:13.974170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:13.974170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:13.974170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:13.974170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:13.974170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:13.974170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:13.974170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:13.974170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:13.974170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:13.974170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:13.974170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:13.974170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:36:13.974170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 6/14] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:36:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:13.979260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba5f0e-9ced-44a4-b4b9-0c830f1665b4 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:13.989035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc0f57-a86d-4df7-88fa-234602746687 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:14.069002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:14.079102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba5f0e-9ced-44a4-b4b9-0c830f1665b4 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:14.081377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:14.088029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc0f57-a86d-4df7-88fa-234602746687 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:14.116590 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f4682-778a-4a2d-a191-8638ee3c10be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:14.117980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f4682-778a-4a2d-a191-8638ee3c10be None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:14.118386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f4682-778a-4a2d-a191-8638ee3c10be None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:14.118697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f4682-778a-4a2d-a191-8638ee3c10be None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:14.119536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f4682-778a-4a2d-a191-8638ee3c10be None admin] RBAC: Adding query filter params (name=admin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:14.119836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f4682-778a-4a2d-a191-8638ee3c10be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:14.121249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f4682-778a-4a2d-a191-8638ee3c10be None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:14.123403 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7c76b-23ae-44c9-939d-8aaf6c33531c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:14.123736 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7c76b-23ae-44c9-939d-8aaf6c33531c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:14.124016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7c76b-23ae-44c9-939d-8aaf6c33531c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:14.124288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7c76b-23ae-44c9-939d-8aaf6c33531c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:14.133470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 9/15] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:14.139144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f4682-778a-4a2d-a191-8638ee3c10be None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:14.191265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7c76b-23ae-44c9-939d-8aaf6c33531c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:14.192721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7c76b-23ae-44c9-939d-8aaf6c33531c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:14.238285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f4682-778a-4a2d-a191-8638ee3c10be None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:14.270037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:14.270340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfa4b2-5c94-449d-b531-73b4e5cdc46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:14.271207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfa4b2-5c94-449d-b531-73b4e5cdc46c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:14.271482 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfa4b2-5c94-449d-b531-73b4e5cdc46c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:14.271754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfa4b2-5c94-449d-b531-73b4e5cdc46c None admin] PATH_INFO: `/v3/users/9947df19bb3b44318e674b5168544c18` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:14.273205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfa4b2-5c94-449d-b531-73b4e5cdc46c None admin] RBAC: Authorizing `identity:get_user(user_id=9947df19bb3b44318e674b5168544c18)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:14.275005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfa4b2-5c94-449d-b531-73b4e5cdc46c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:14.276555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 10/16] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:14 2022] GET /identity/v3/users/9947df19bb3b44318e674b5168544c18 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:14.282114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfa4b2-5c94-449d-b531-73b4e5cdc46c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:14.288238 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 7/17] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 12:36:14.294269 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7c76b-23ae-44c9-939d-8aaf6c33531c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:14.318288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:14.384173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfa4b2-5c94-449d-b531-73b4e5cdc46c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:14.392647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7c76b-23ae-44c9-939d-8aaf6c33531c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:14.417401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35a78-9378-4518-9d21-f05d029ea019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:14.418034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35a78-9378-4518-9d21-f05d029ea019 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:14.418315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35a78-9378-4518-9d21-f05d029ea019 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:14.418623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35a78-9378-4518-9d21-f05d029ea019 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:14.419771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b35a78-9378-4518-9d21-f05d029ea019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:14.421210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b35a78-9378-4518-9d21-f05d029ea019 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:14.422782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 11/18] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:14.426145 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cab057-54eb-4463-92fd-d4f085432d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:14.426798 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cab057-54eb-4463-92fd-d4f085432d86 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:14.427083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cab057-54eb-4463-92fd-d4f085432d86 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:14.427343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cab057-54eb-4463-92fd-d4f085432d86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:14.428564 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35a78-9378-4518-9d21-f05d029ea019 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:14.437667 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cab057-54eb-4463-92fd-d4f085432d86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:14.439183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cab057-54eb-4463-92fd-d4f085432d86 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-98cab057-54eb-4463-92fd-d4f085432d86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:36:14.456375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:14.457270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:36:14.458093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:36:14.458093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:36:14.458093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:36:14.458093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 8/19] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:36:14.463705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cab057-54eb-4463-92fd-d4f085432d86 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:14.497476 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:14.526794 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35a78-9378-4518-9d21-f05d029ea019 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:14.528331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:14.560402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779d44b-ee90-483e-a6db-14871f16948d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:14.561004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779d44b-ee90-483e-a6db-14871f16948d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:14.561288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779d44b-ee90-483e-a6db-14871f16948d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:14.561561 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779d44b-ee90-483e-a6db-14871f16948d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:14.562284 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779d44b-ee90-483e-a6db-14871f16948d None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.domain.id=default, user.id=9947df19bb3b44318e674b5168544c18) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:14.562682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779d44b-ee90-483e-a6db-14871f16948d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:14.562857 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cab057-54eb-4463-92fd-d4f085432d86 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:14.564140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779d44b-ee90-483e-a6db-14871f16948d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:14.574788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 12/20] 10.222.0.46 () {56 vars in 1322 bytes} [Fri Sep 9 12:36:14 2022] GET /identity/v3/role_assignments?user.id=9947df19bb3b44318e674b5168544c18&scope.domain.id=default&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:14.597836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e207a-1503-492b-981f-e0d5bfa7e2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:14.598462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e207a-1503-492b-981f-e0d5bfa7e2d7 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:14.598783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e207a-1503-492b-981f-e0d5bfa7e2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:14.599059 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e207a-1503-492b-981f-e0d5bfa7e2d7 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:14.599720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e207a-1503-492b-981f-e0d5bfa7e2d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:14.600028 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e207a-1503-492b-981f-e0d5bfa7e2d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:14.601478 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e207a-1503-492b-981f-e0d5bfa7e2d7 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:14.621026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 9/21] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:17.186265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b8aba-253a-46ee-b422-a61c9e7a1b01 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:17.187164 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b8aba-253a-46ee-b422-a61c9e7a1b01 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:17.187609 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b8aba-253a-46ee-b422-a61c9e7a1b01 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:17.189497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 13/22] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:17.195753 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08921e4-af05-4446-b0f6-11ae0c2e429a None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:17.196635 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08921e4-af05-4446-b0f6-11ae0c2e429a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:17.197052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08921e4-af05-4446-b0f6-11ae0c2e429a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:17.218241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-f08921e4-af05-4446-b0f6-11ae0c2e429a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:17.223777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f08921e4-af05-4446-b0f6-11ae0c2e429a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:17.309005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 10/23] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:17.356300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdf1a1-24b9-4d41-bdc4-71419ba743a6 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:17.356987 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdf1a1-24b9-4d41-bdc4-71419ba743a6 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:17.357472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdf1a1-24b9-4d41-bdc4-71419ba743a6 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:17.359538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 14/24] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:17.365829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e454e2d-44a0-4f5d-99ce-cfbbb107aa2a None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:17.366119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e454e2d-44a0-4f5d-99ce-cfbbb107aa2a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:17.366382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e454e2d-44a0-4f5d-99ce-cfbbb107aa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:17.391347 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-2e454e2d-44a0-4f5d-99ce-cfbbb107aa2a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:17.392883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2e454e2d-44a0-4f5d-99ce-cfbbb107aa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:17.417459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:17.485001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 11/25] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:17.535157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7b707-1e33-4747-ad89-841b969a6304 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:17.535483 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7b707-1e33-4747-ad89-841b969a6304 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:17.535719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7b707-1e33-4747-ad89-841b969a6304 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:17.537311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 15/26] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:17.547052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44335f-9780-42af-8c0b-c6ca38f10fc9 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:17.547343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44335f-9780-42af-8c0b-c6ca38f10fc9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:17.547624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44335f-9780-42af-8c0b-c6ca38f10fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:17.570794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-2e44335f-9780-42af-8c0b-c6ca38f10fc9 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:17.571974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2e44335f-9780-42af-8c0b-c6ca38f10fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:17.626067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4d6db-fed4-4175-b406-08ebe6f56439 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:17.626719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4d6db-fed4-4175-b406-08ebe6f56439 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:17.627194 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4d6db-fed4-4175-b406-08ebe6f56439 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:17.629225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 16/27] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:36:17.634716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a912-3008-432c-a928-fbc2f18c3016 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:17.635264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a912-3008-432c-a928-fbc2f18c3016 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:17.635774 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a912-3008-432c-a928-fbc2f18c3016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:17.666897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 12/28] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:17.675510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-6b30a912-3008-432c-a928-fbc2f18c3016 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:17.676983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6b30a912-3008-432c-a928-fbc2f18c3016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:17.734724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44335f-9780-42af-8c0b-c6ca38f10fc9 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:17.813156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 17/29] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:17.864539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:17.882791 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30a912-3008-432c-a928-fbc2f18c3016 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:17.886739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44335f-9780-42af-8c0b-c6ca38f10fc9 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:17.925962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6acf15-eb7d-4c8d-9f5f-6e4cefb2cc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:17.926522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6acf15-eb7d-4c8d-9f5f-6e4cefb2cc35 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:17.926809 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6acf15-eb7d-4c8d-9f5f-6e4cefb2cc35 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:17.927083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6acf15-eb7d-4c8d-9f5f-6e4cefb2cc35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:17.946645 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6acf15-eb7d-4c8d-9f5f-6e4cefb2cc35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-7e6acf15-eb7d-4c8d-9f5f-6e4cefb2cc35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:17.967547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:17.968445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:17.969309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:17.969309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:17.969309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:17.969309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:17.969309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:17.969309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:17.969309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:17.969309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:17.969309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:17.969309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:17.969309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:17.969309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:36:17.969309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 13/30] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:36:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:17.988543 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6acf15-eb7d-4c8d-9f5f-6e4cefb2cc35 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:18.026470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:18.032957 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30a912-3008-432c-a928-fbc2f18c3016 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:18.080048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad08723-4866-4303-b92f-a30201e56ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:18.080685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad08723-4866-4303-b92f-a30201e56ad5 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:18.081012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad08723-4866-4303-b92f-a30201e56ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:18.081226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad08723-4866-4303-b92f-a30201e56ad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:18.115539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad08723-4866-4303-b92f-a30201e56ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:18.116988 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad08723-4866-4303-b92f-a30201e56ad5 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-0ad08723-4866-4303-b92f-a30201e56ad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:36:18.147665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:18.150319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:36:18.152161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:36:18.152161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:36:18.152161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:18.152161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 18/31] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:36:18.152161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad08723-4866-4303-b92f-a30201e56ad5 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:18.159444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6acf15-eb7d-4c8d-9f5f-6e4cefb2cc35 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:18.206592 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67766521-d2ef-41eb-8c6b-8fe267502a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:18.207255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67766521-d2ef-41eb-8c6b-8fe267502a5d None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:18.207391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67766521-d2ef-41eb-8c6b-8fe267502a5d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:18.207653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67766521-d2ef-41eb-8c6b-8fe267502a5d None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:18.208317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67766521-d2ef-41eb-8c6b-8fe267502a5d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:18.210343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67766521-d2ef-41eb-8c6b-8fe267502a5d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:18.259631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad08723-4866-4303-b92f-a30201e56ad5 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:18.295387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a5a5c-8cb9-493e-b13c-75e40d2837c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:18.296003 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a5a5c-8cb9-493e-b13c-75e40d2837c9 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:18.296579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a5a5c-8cb9-493e-b13c-75e40d2837c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:18.296864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a5a5c-8cb9-493e-b13c-75e40d2837c9 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:18.297538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a5a5c-8cb9-493e-b13c-75e40d2837c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:18.297841 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a5a5c-8cb9-493e-b13c-75e40d2837c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:18.299244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a5a5c-8cb9-493e-b13c-75e40d2837c9 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:18.309366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 19/32] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:18.310166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a5a5c-8cb9-493e-b13c-75e40d2837c9 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:18.326706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:18.420858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a5a5c-8cb9-493e-b13c-75e40d2837c9 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:18.444927 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 14/33] 10.222.0.46 () {60 vars in 1131 bytes} [Fri Sep 9 12:36:17 2022] POST /identity/v3/roles => generated 219 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:36:18.447076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67766521-d2ef-41eb-8c6b-8fe267502a5d None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:18.451720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:18.474416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba330a-5631-4cdc-be83-fdb37ea1216a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:18.475085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba330a-5631-4cdc-be83-fdb37ea1216a None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:18.475365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba330a-5631-4cdc-be83-fdb37ea1216a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:18.477510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba330a-5631-4cdc-be83-fdb37ea1216a None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:18.478288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba330a-5631-4cdc-be83-fdb37ea1216a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:18.479759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba330a-5631-4cdc-be83-fdb37ea1216a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:18.586720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67766521-d2ef-41eb-8c6b-8fe267502a5d None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:18.626099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705587aa-2dd3-4608-8865-5ce7dc31c1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:18.626822 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705587aa-2dd3-4608-8865-5ce7dc31c1da None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:18.627163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705587aa-2dd3-4608-8865-5ce7dc31c1da None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:18.627442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705587aa-2dd3-4608-8865-5ce7dc31c1da None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:18.628162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705587aa-2dd3-4608-8865-5ce7dc31c1da None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:18.628469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705587aa-2dd3-4608-8865-5ce7dc31c1da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:18.630143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705587aa-2dd3-4608-8865-5ce7dc31c1da None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:18.644518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 15/34] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:18.645442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705587aa-2dd3-4608-8865-5ce7dc31c1da None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:18.720124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:18.751180 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 20/35] 10.222.0.46 () {60 vars in 1131 bytes} [Fri Sep 9 12:36:18 2022] POST /identity/v3/roles => generated 225 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:36:18.751904 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba330a-5631-4cdc-be83-fdb37ea1216a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:18.765082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705587aa-2dd3-4608-8865-5ce7dc31c1da None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:18.817439 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0781a3-bd98-4472-a239-8a5515930ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:18.818470 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0781a3-bd98-4472-a239-8a5515930ba5 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:18.818900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0781a3-bd98-4472-a239-8a5515930ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:18.819308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0781a3-bd98-4472-a239-8a5515930ba5 None admin] PATH_INFO: `/v3/users/9947df19bb3b44318e674b5168544c18` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:18.821671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0781a3-bd98-4472-a239-8a5515930ba5 None admin] RBAC: Authorizing `identity:get_user(user_id=9947df19bb3b44318e674b5168544c18)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:18.823506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0781a3-bd98-4472-a239-8a5515930ba5 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:18.825387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 16/36] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:18 2022] GET /identity/v3/users/9947df19bb3b44318e674b5168544c18 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:18.832167 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0781a3-bd98-4472-a239-8a5515930ba5 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:18.840488 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:18.885060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba330a-5631-4cdc-be83-fdb37ea1216a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:18.933295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51707444-bd8c-4e12-ae89-9ab41d54c2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:18.933951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51707444-bd8c-4e12-ae89-9ab41d54c2b1 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:18.934231 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51707444-bd8c-4e12-ae89-9ab41d54c2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:18.934522 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51707444-bd8c-4e12-ae89-9ab41d54c2b1 None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:18.935215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51707444-bd8c-4e12-ae89-9ab41d54c2b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:18.936803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51707444-bd8c-4e12-ae89-9ab41d54c2b1 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:19.069219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0781a3-bd98-4472-a239-8a5515930ba5 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:19.072330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 21/37] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:36:18 2022] POST /identity/v3/projects => generated 301 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:36:19.074250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb0113-6f58-424e-963d-d08d4dacef68 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:19.074250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb0113-6f58-424e-963d-d08d4dacef68 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:19.074250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb0113-6f58-424e-963d-d08d4dacef68 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:19.078578 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 22/38] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:19.082246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f25c86-9ab5-4e11-96ce-cfc70234175c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:19.082559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f25c86-9ab5-4e11-96ce-cfc70234175c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:19.082836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f25c86-9ab5-4e11-96ce-cfc70234175c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:19.108849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-67f25c86-9ab5-4e11-96ce-cfc70234175c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:19.110672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-67f25c86-9ab5-4e11-96ce-cfc70234175c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:19.113502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:19.125174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d554c-2e3c-4935-9492-6d45cbff10ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:19.126358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d554c-2e3c-4935-9492-6d45cbff10ff None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:19.126358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d554c-2e3c-4935-9492-6d45cbff10ff None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:19.126358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d554c-2e3c-4935-9492-6d45cbff10ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:19.128630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d554c-2e3c-4935-9492-6d45cbff10ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:19.130343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d554c-2e3c-4935-9492-6d45cbff10ff None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:19.134748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 17/39] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:19.145528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d554c-2e3c-4935-9492-6d45cbff10ff None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:19.246104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 23/40] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:19.264175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bc627-a68e-4004-b3d9-92276d9de581 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:19.264770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bc627-a68e-4004-b3d9-92276d9de581 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:19.264770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bc627-a68e-4004-b3d9-92276d9de581 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:19.266293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 24/41] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:36:19.271815 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4452f-c846-4b74-9839-f0a3180ae358 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:19.272101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4452f-c846-4b74-9839-f0a3180ae358 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:19.272384 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4452f-c846-4b74-9839-f0a3180ae358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:19.289445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d554c-2e3c-4935-9492-6d45cbff10ff None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:19.291827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d6c4452f-c846-4b74-9839-f0a3180ae358 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:19.293314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d6c4452f-c846-4b74-9839-f0a3180ae358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:19.333707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633f14b-bf86-48e2-b3f1-288929f67bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:19.334410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633f14b-bf86-48e2-b3f1-288929f67bb8 None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:19.334824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633f14b-bf86-48e2-b3f1-288929f67bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:19.335155 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633f14b-bf86-48e2-b3f1-288929f67bb8 None admin] PATH_INFO: `/v3/domains/default/users/9947df19bb3b44318e674b5168544c18/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:19.338166 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633f14b-bf86-48e2-b3f1-288929f67bb8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9947df19bb3b44318e674b5168544c18, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:19.339897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633f14b-bf86-48e2-b3f1-288929f67bb8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:19.355502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:19.377643 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 25/42] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:19.406959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 18/43] 10.222.0.46 () {58 vars in 1361 bytes} [Fri Sep 9 12:36:19 2022] PUT /identity/v3/domains/default/users/9947df19bb3b44318e674b5168544c18/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:36:19.550594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4452f-c846-4b74-9839-f0a3180ae358 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:19.654944 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633f14b-bf86-48e2-b3f1-288929f67bb8 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:19.657173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4452f-c846-4b74-9839-f0a3180ae358 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:19.704478 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e62cc-2d9f-4828-aed8-be30b94039b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:19.705563 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e62cc-2d9f-4828-aed8-be30b94039b2 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:19.706336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e62cc-2d9f-4828-aed8-be30b94039b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:19.706854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e62cc-2d9f-4828-aed8-be30b94039b2 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:19.707724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e62cc-2d9f-4828-aed8-be30b94039b2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:19.709692 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e62cc-2d9f-4828-aed8-be30b94039b2 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:19.715284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:19.755340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633f14b-bf86-48e2-b3f1-288929f67bb8 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:19.787858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54815838-6d20-4a25-a714-8b78282baf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:19.788496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54815838-6d20-4a25-a714-8b78282baf6c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:19.788779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54815838-6d20-4a25-a714-8b78282baf6c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:19.789055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54815838-6d20-4a25-a714-8b78282baf6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:19.791025 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54815838-6d20-4a25-a714-8b78282baf6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:19.792480 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54815838-6d20-4a25-a714-8b78282baf6c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:19.794056 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 19/44] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:19.799541 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54815838-6d20-4a25-a714-8b78282baf6c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:19.854525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 26/45] 10.222.0.46 () {60 vars in 1131 bytes} [Fri Sep 9 12:36:19 2022] POST /identity/v3/roles => generated 223 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:36:19.912780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54815838-6d20-4a25-a714-8b78282baf6c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:19.941910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:19.951067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c58743-83df-46f4-9c37-1e0a3a974232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:19.952526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c58743-83df-46f4-9c37-1e0a3a974232 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:19.952526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c58743-83df-46f4-9c37-1e0a3a974232 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:19.952526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c58743-83df-46f4-9c37-1e0a3a974232 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:19.953021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c58743-83df-46f4-9c37-1e0a3a974232 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:19.954705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c58743-83df-46f4-9c37-1e0a3a974232 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:20.064661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 20/46] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:36:19 2022] POST /identity/v3/projects => generated 312 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:36:21.671823 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fa685-ace4-4746-ab68-c82d63ed17a2 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:21.672342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fa685-ace4-4746-ab68-c82d63ed17a2 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:21.672426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fa685-ace4-4746-ab68-c82d63ed17a2 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:21.673745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 27/47] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:21.679253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7d954-5853-4749-8550-368eeed6a820 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:21.679614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7d954-5853-4749-8550-368eeed6a820 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:21.680479 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7d954-5853-4749-8550-368eeed6a820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:21.699544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-bde7d954-5853-4749-8550-368eeed6a820 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:21.701087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-bde7d954-5853-4749-8550-368eeed6a820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:21.766766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 21/48] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:22.034428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6fa685-ace4-4746-ab68-c82d63ed17a2 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:22.078585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:22.137852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6fa685-ace4-4746-ab68-c82d63ed17a2 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:22.171505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f55f2-2797-420b-9cfe-eb941fde6c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:22.172130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f55f2-2797-420b-9cfe-eb941fde6c1a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:22.172517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f55f2-2797-420b-9cfe-eb941fde6c1a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:22.172795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f55f2-2797-420b-9cfe-eb941fde6c1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:22.184922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f55f2-2797-420b-9cfe-eb941fde6c1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-f27f55f2-2797-420b-9cfe-eb941fde6c1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:22.190775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:22.191669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:22.192435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:22.192435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:22.192435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:22.192435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:22.192435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:22.192435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:22.192435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:22.192435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:22.192435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:22.192435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:22.192435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:22.192435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:22.192435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 28/49] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:36:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:22.198665 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7d954-5853-4749-8550-368eeed6a820 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:22.243665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577144b-3b79-4d86-8487-1c5f77ac68d6 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:22.243946 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577144b-3b79-4d86-8487-1c5f77ac68d6 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:22.244222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577144b-3b79-4d86-8487-1c5f77ac68d6 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:22.245611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 29/50] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:22.250465 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cba64-6c4e-42c1-a601-c055567478a6 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:22.250780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cba64-6c4e-42c1-a601-c055567478a6 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:22.251066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cba64-6c4e-42c1-a601-c055567478a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:22.269162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-689cba64-6c4e-42c1-a601-c055567478a6 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:22.270361 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-689cba64-6c4e-42c1-a601-c055567478a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:22.298167 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7d954-5853-4749-8550-368eeed6a820 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:22.309864 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:22.332625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e097e4-45a4-4f3b-a992-43edda128d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:22.333232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e097e4-45a4-4f3b-a992-43edda128d8e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:22.333514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e097e4-45a4-4f3b-a992-43edda128d8e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:22.333785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e097e4-45a4-4f3b-a992-43edda128d8e None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:22.334441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e097e4-45a4-4f3b-a992-43edda128d8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:22.334773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e097e4-45a4-4f3b-a992-43edda128d8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:22.336176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e097e4-45a4-4f3b-a992-43edda128d8e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:22.336274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 30/51] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:22.348264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 22/52] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:22.354156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cba64-6c4e-42c1-a601-c055567478a6 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:22.384200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:22.449250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cba64-6c4e-42c1-a601-c055567478a6 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:22.481102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556573d-6c49-4ed5-9394-7326f9e38d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:22.481721 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556573d-6c49-4ed5-9394-7326f9e38d63 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:22.481999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556573d-6c49-4ed5-9394-7326f9e38d63 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:22.482267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556573d-6c49-4ed5-9394-7326f9e38d63 None admin] PATH_INFO: `/v3/users/9947df19bb3b44318e674b5168544c18` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:22.483413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556573d-6c49-4ed5-9394-7326f9e38d63 None admin] RBAC: Authorizing `identity:get_user(user_id=9947df19bb3b44318e674b5168544c18)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:22.484838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556573d-6c49-4ed5-9394-7326f9e38d63 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:22.486371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 31/53] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:22 2022] GET /identity/v3/users/9947df19bb3b44318e674b5168544c18 => generated 247 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:22.491732 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e097e4-45a4-4f3b-a992-43edda128d8e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:22.589482 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e097e4-45a4-4f3b-a992-43edda128d8e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:22.601362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556573d-6c49-4ed5-9394-7326f9e38d63 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:22.623081 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b157f6-f23a-4015-8305-61af0fd83990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:22.623718 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b157f6-f23a-4015-8305-61af0fd83990 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:22.624001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b157f6-f23a-4015-8305-61af0fd83990 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:22.624275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b157f6-f23a-4015-8305-61af0fd83990 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:22.625378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b157f6-f23a-4015-8305-61af0fd83990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:22.626873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b157f6-f23a-4015-8305-61af0fd83990 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:22.628425 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 23/54] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:22.635007 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b157f6-f23a-4015-8305-61af0fd83990 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:22.676637 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:22.698221 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:22.701430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556573d-6c49-4ed5-9394-7326f9e38d63 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:22.732625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b157f6-f23a-4015-8305-61af0fd83990 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:22.735028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4331ac-c819-4134-8667-10b93cc25a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:22.735664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4331ac-c819-4134-8667-10b93cc25a5a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:22.735948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4331ac-c819-4134-8667-10b93cc25a5a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:22.736224 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4331ac-c819-4134-8667-10b93cc25a5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:22.737434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4331ac-c819-4134-8667-10b93cc25a5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:22.738916 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4331ac-c819-4134-8667-10b93cc25a5a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:22.740498 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 32/55] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:22.746665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4331ac-c819-4134-8667-10b93cc25a5a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:22.767713 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c653286-099f-4d9b-bf9b-df3a4af21346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:22.768330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c653286-099f-4d9b-bf9b-df3a4af21346 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:22.768622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c653286-099f-4d9b-bf9b-df3a4af21346 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:22.768900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c653286-099f-4d9b-bf9b-df3a4af21346 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:22.769619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c653286-099f-4d9b-bf9b-df3a4af21346 None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.domain.id=default, user.id=9947df19bb3b44318e674b5168544c18) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:22.769929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c653286-099f-4d9b-bf9b-df3a4af21346 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:22.771443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c653286-099f-4d9b-bf9b-df3a4af21346 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:22.778283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 24/56] 10.222.0.46 () {56 vars in 1322 bytes} [Fri Sep 9 12:36:22 2022] GET /identity/v3/role_assignments?user.id=9947df19bb3b44318e674b5168544c18&scope.domain.id=default&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:22.849110 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4331ac-c819-4134-8667-10b93cc25a5a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:22.885379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3251e-a277-4486-a382-663c2746d6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:22.885996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3251e-a277-4486-a382-663c2746d6e5 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:22.886274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3251e-a277-4486-a382-663c2746d6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:22.886575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3251e-a277-4486-a382-663c2746d6e5 None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:22.887233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3251e-a277-4486-a382-663c2746d6e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:22.888692 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3251e-a277-4486-a382-663c2746d6e5 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:22.968295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 33/57] 10.222.0.46 () {60 vars in 1140 bytes} [Fri Sep 9 12:36:22 2022] POST /identity/v3/projects => generated 298 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:36:25.099529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21751450-1d66-4471-a439-80bd43b340ad None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:25.100048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21751450-1d66-4471-a439-80bd43b340ad None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:25.100130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21751450-1d66-4471-a439-80bd43b340ad None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:25.101448 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 25/58] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:25.107017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18e4d5-99e7-4456-88b1-8c4755f06596 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:25.107415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18e4d5-99e7-4456-88b1-8c4755f06596 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:25.107569 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18e4d5-99e7-4456-88b1-8c4755f06596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:25.126810 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-5e18e4d5-99e7-4456-88b1-8c4755f06596 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:25.127989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5e18e4d5-99e7-4456-88b1-8c4755f06596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:25.142179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:25.297160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 34/59] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:25.562147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21751450-1d66-4471-a439-80bd43b340ad None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:25.659880 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21751450-1d66-4471-a439-80bd43b340ad None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:25.662080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:25.691542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c33d7-1a45-439a-bfd3-8bd89b55b2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:25.692164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c33d7-1a45-439a-bfd3-8bd89b55b2ab None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:25.692445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c33d7-1a45-439a-bfd3-8bd89b55b2ab None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:25.692717 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c33d7-1a45-439a-bfd3-8bd89b55b2ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:25.693804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c33d7-1a45-439a-bfd3-8bd89b55b2ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:25.695272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c33d7-1a45-439a-bfd3-8bd89b55b2ab None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:25.697505 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 26/60] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:25.702764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18e4d5-99e7-4456-88b1-8c4755f06596 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:25.796972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18e4d5-99e7-4456-88b1-8c4755f06596 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:25.831090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469e599-831c-4f2b-b932-f5c523a7b9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:25.831631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469e599-831c-4f2b-b932-f5c523a7b9b0 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:25.831910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469e599-831c-4f2b-b932-f5c523a7b9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:25.832179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469e599-831c-4f2b-b932-f5c523a7b9b0 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:25.832826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469e599-831c-4f2b-b932-f5c523a7b9b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:25.834302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469e599-831c-4f2b-b932-f5c523a7b9b0 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:25.932933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 35/61] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:36:25 2022] POST /identity/v3/users => generated 275 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:36:29.341342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fece7c-1ce8-4c87-8d8f-e1949a4d249a None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:29.341850 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fece7c-1ce8-4c87-8d8f-e1949a4d249a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:29.341923 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fece7c-1ce8-4c87-8d8f-e1949a4d249a None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:29.343281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 27/62] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:29.352584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25cde4-54cd-4e08-92ee-457c4397e025 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:29.353210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25cde4-54cd-4e08-92ee-457c4397e025 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:29.353210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25cde4-54cd-4e08-92ee-457c4397e025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:29.370031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:29.380536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-cb25cde4-54cd-4e08-92ee-457c4397e025 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:29.381675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-cb25cde4-54cd-4e08-92ee-457c4397e025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:29.449920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced83ac-b261-4c78-84e7-32cd835312cf None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:29.450205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced83ac-b261-4c78-84e7-32cd835312cf None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:29.450671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced83ac-b261-4c78-84e7-32cd835312cf None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:29.452410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 28/63] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:29.472324 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 36/64] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:29.474416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da172567-12d6-4ce8-848b-9cb5c9e6df7c None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:29.474972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da172567-12d6-4ce8-848b-9cb5c9e6df7c None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:29.475444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da172567-12d6-4ce8-848b-9cb5c9e6df7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:29.506752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-da172567-12d6-4ce8-848b-9cb5c9e6df7c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:29.508367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-da172567-12d6-4ce8-848b-9cb5c9e6df7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:29.512892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc97108-810a-4272-a8e9-c19a1b7926cc None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:29.513042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc97108-810a-4272-a8e9-c19a1b7926cc None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:29.513279 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc97108-810a-4272-a8e9-c19a1b7926cc None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:29.515074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 37/65] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:36:29.521213 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8460f-840b-4a5c-984d-88f3d6beadbf None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:29.521502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8460f-840b-4a5c-984d-88f3d6beadbf None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:29.521788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8460f-840b-4a5c-984d-88f3d6beadbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:29.558454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-cca8460f-840b-4a5c-984d-88f3d6beadbf None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:29.559664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-cca8460f-840b-4a5c-984d-88f3d6beadbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:29.618745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 29/66] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:29.684613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 38/67] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:29.779184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da172567-12d6-4ce8-848b-9cb5c9e6df7c None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:29.804274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f7c70-d8fc-4b14-a35d-e615969db8fd None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:29.805408 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f7c70-d8fc-4b14-a35d-e615969db8fd None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:29.806128 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f7c70-d8fc-4b14-a35d-e615969db8fd None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:29.808471 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 39/68] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:36:29.818871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e8836-649b-4c9c-b813-138a50fba24b None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:29.819401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e8836-649b-4c9c-b813-138a50fba24b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:29.820111 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e8836-649b-4c9c-b813-138a50fba24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:29.852827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-2e9e8836-649b-4c9c-b813-138a50fba24b None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:29.854251 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2e9e8836-649b-4c9c-b813-138a50fba24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:29.897597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da172567-12d6-4ce8-848b-9cb5c9e6df7c None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:29.899683 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:29.918977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:29.926566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 40/69] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:29.927800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc869b-7faa-4993-98cd-26d635bbf93c None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:29.928121 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc869b-7faa-4993-98cd-26d635bbf93c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:29.928428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc869b-7faa-4993-98cd-26d635bbf93c None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:29.929816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 41/70] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:29.931202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc869b-7faa-4993-98cd-26d635bbf93c None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:29.958447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cc0e1-5bf0-4809-b6ce-fa84a4f1e7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:29.959299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cc0e1-5bf0-4809-b6ce-fa84a4f1e7e1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:29.959586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cc0e1-5bf0-4809-b6ce-fa84a4f1e7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:29.959865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cc0e1-5bf0-4809-b6ce-fa84a4f1e7e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:29.975630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cc0e1-5bf0-4809-b6ce-fa84a4f1e7e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-804cc0e1-5bf0-4809-b6ce-fa84a4f1e7e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:29.982976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:29.983998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:29.984773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:29.984773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:29.984773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:29.984773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:29.984773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:29.984773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:29.984773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:29.984773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:29.984773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:29.984773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:29.984773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:29.984773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:36:29.984773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 30/71] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:29.985395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00f945-45f1-40f4-bbaf-f426a9a60aaa None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:29.985574 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00f945-45f1-40f4-bbaf-f426a9a60aaa None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:29.985858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00f945-45f1-40f4-bbaf-f426a9a60aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:30.012089 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ac00f945-45f1-40f4-bbaf-f426a9a60aaa None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:30.014215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ac00f945-45f1-40f4-bbaf-f426a9a60aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:30.049018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc869b-7faa-4993-98cd-26d635bbf93c None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:30.084365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d870d-5973-4e5f-89db-54825ace2a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:30.084990 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d870d-5973-4e5f-89db-54825ace2a4e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:30.085267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d870d-5973-4e5f-89db-54825ace2a4e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:30.085593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d870d-5973-4e5f-89db-54825ace2a4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:30.087473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 31/72] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:30.088311 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00f945-45f1-40f4-bbaf-f426a9a60aaa None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:30.111917 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d870d-5973-4e5f-89db-54825ace2a4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-226d870d-5973-4e5f-89db-54825ace2a4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:30.120428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:30.121318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 42/73] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:36:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:30.122092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d870d-5973-4e5f-89db-54825ace2a4e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:30.200368 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:30.204386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:30.209506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00f945-45f1-40f4-bbaf-f426a9a60aaa None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:30.244672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77416723-7809-472f-9fca-c50e356916f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:30.245288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416723-7809-472f-9fca-c50e356916f8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:30.245837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416723-7809-472f-9fca-c50e356916f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:30.246336 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416723-7809-472f-9fca-c50e356916f8 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:30.247011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77416723-7809-472f-9fca-c50e356916f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:30.247569 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77416723-7809-472f-9fca-c50e356916f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:30.249333 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77416723-7809-472f-9fca-c50e356916f8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:30.250375 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d870d-5973-4e5f-89db-54825ace2a4e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:30.258070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 32/74] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:30.258855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77416723-7809-472f-9fca-c50e356916f8 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:30.288035 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69a9c5-e978-4415-84d7-b8de6370a436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:30.288664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69a9c5-e978-4415-84d7-b8de6370a436 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:30.288940 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69a9c5-e978-4415-84d7-b8de6370a436 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:30.289854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69a9c5-e978-4415-84d7-b8de6370a436 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:30.300278 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69a9c5-e978-4415-84d7-b8de6370a436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-9b69a9c5-e978-4415-84d7-b8de6370a436 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:30.308677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:30.309766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 43/75] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:36:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:30.310772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69a9c5-e978-4415-84d7-b8de6370a436 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:30.379056 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77416723-7809-472f-9fca-c50e356916f8 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:30.411568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd7ea7-1a24-4729-9b4b-08c2c45840de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:30.412178 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd7ea7-1a24-4729-9b4b-08c2c45840de None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:30.412462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd7ea7-1a24-4729-9b4b-08c2c45840de None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:30.412736 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd7ea7-1a24-4729-9b4b-08c2c45840de None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:30.413394 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd7ea7-1a24-4729-9b4b-08c2c45840de None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:30.413701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd7ea7-1a24-4729-9b4b-08c2c45840de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:30.415688 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd7ea7-1a24-4729-9b4b-08c2c45840de None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:30.419326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69a9c5-e978-4415-84d7-b8de6370a436 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:30.423486 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 33/76] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:30.424268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd7ea7-1a24-4729-9b4b-08c2c45840de None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:30.435712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:30.446587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:30.453072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e4cea-5fdc-4dc8-b33c-3e80af8f0bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:30.453709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e4cea-5fdc-4dc8-b33c-3e80af8f0bad None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:30.453997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e4cea-5fdc-4dc8-b33c-3e80af8f0bad None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:30.454276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e4cea-5fdc-4dc8-b33c-3e80af8f0bad None admin] PATH_INFO: `/v3/roles/member` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:30.464699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e4cea-5fdc-4dc8-b33c-3e80af8f0bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-d32e4cea-5fdc-4dc8-b33c-3e80af8f0bad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:30.469972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:30.470936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:30.471880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:30.471880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:30.471880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:30.471880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:30.471880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:30.471880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:30.471880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:30.471880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:30.471880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:30.471880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:30.471880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:30.471880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:30.472865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 44/77] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:30.473663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e4cea-5fdc-4dc8-b33c-3e80af8f0bad None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:30.528927 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd7ea7-1a24-4729-9b4b-08c2c45840de None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:30.562331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc362943-b056-4ff4-bd82-a8b734e8c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:30.562990 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc362943-b056-4ff4-bd82-a8b734e8c073 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:30.563277 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc362943-b056-4ff4-bd82-a8b734e8c073 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:30.563559 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc362943-b056-4ff4-bd82-a8b734e8c073 None admin] PATH_INFO: `/v3/users/4694cfe2faf74f7085cc07187b0f047f` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:30.570375 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e4cea-5fdc-4dc8-b33c-3e80af8f0bad None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:30.578414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc362943-b056-4ff4-bd82-a8b734e8c073 None admin] RBAC: Authorizing `identity:get_user(user_id=4694cfe2faf74f7085cc07187b0f047f)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:30.580289 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc362943-b056-4ff4-bd82-a8b734e8c073 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:30.582233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 34/78] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:30 2022] GET /identity/v3/users/4694cfe2faf74f7085cc07187b0f047f => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:30.583037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc362943-b056-4ff4-bd82-a8b734e8c073 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:30.607183 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd76d6-bde5-413c-94cc-024678f00040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:30.607793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd76d6-bde5-413c-94cc-024678f00040 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:30.608070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd76d6-bde5-413c-94cc-024678f00040 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:30.608339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd76d6-bde5-413c-94cc-024678f00040 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:30.609005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd76d6-bde5-413c-94cc-024678f00040 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:30.609306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd76d6-bde5-413c-94cc-024678f00040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:30.610835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd76d6-bde5-413c-94cc-024678f00040 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:30.619754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 45/79] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:36:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:30.620566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd76d6-bde5-413c-94cc-024678f00040 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:30.660741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:30.666090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:30.685187 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc362943-b056-4ff4-bd82-a8b734e8c073 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:30.717586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f2d78-801f-43f9-a1cd-501b6f4bac5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:30.718143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f2d78-801f-43f9-a1cd-501b6f4bac5a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:30.718425 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f2d78-801f-43f9-a1cd-501b6f4bac5a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:30.718728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f2d78-801f-43f9-a1cd-501b6f4bac5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:30.719874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f2d78-801f-43f9-a1cd-501b6f4bac5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:30.721320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f2d78-801f-43f9-a1cd-501b6f4bac5a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:30.722894 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 35/80] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:30.723651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f2d78-801f-43f9-a1cd-501b6f4bac5a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:30.723769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd76d6-bde5-413c-94cc-024678f00040 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:30.760084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00fd8a9-32bb-44fd-bf75-a05640f7764d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:30.760716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fd8a9-32bb-44fd-bf75-a05640f7764d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:30.760998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fd8a9-32bb-44fd-bf75-a05640f7764d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:30.761276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fd8a9-32bb-44fd-bf75-a05640f7764d None admin] PATH_INFO: `/v3/users/9947df19bb3b44318e674b5168544c18` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:30.762408 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00fd8a9-32bb-44fd-bf75-a05640f7764d None admin] RBAC: Authorizing `identity:get_user(user_id=9947df19bb3b44318e674b5168544c18)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:30.763935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00fd8a9-32bb-44fd-bf75-a05640f7764d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:30.765521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 46/81] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:30 2022] GET /identity/v3/users/9947df19bb3b44318e674b5168544c18 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:30.766284 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00fd8a9-32bb-44fd-bf75-a05640f7764d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:30.825140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f2d78-801f-43f9-a1cd-501b6f4bac5a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:30.859485 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f22a1-fb47-43bb-913f-c2adcf4992a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:30.860092 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f22a1-fb47-43bb-913f-c2adcf4992a1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:30.860378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f22a1-fb47-43bb-913f-c2adcf4992a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:30.860656 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f22a1-fb47-43bb-913f-c2adcf4992a1 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:30.861314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f22a1-fb47-43bb-913f-c2adcf4992a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:30.861619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f22a1-fb47-43bb-913f-c2adcf4992a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:30.863025 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f22a1-fb47-43bb-913f-c2adcf4992a1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:30.865540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00fd8a9-32bb-44fd-bf75-a05640f7764d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:30.871699 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 36/82] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:30.872508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f22a1-fb47-43bb-913f-c2adcf4992a1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:30.873116 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:30.875847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:30.900787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e133eb8-7b2c-4eed-bc87-589910c7118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:30.901385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e133eb8-7b2c-4eed-bc87-589910c7118c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:30.901663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e133eb8-7b2c-4eed-bc87-589910c7118c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:30.901935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e133eb8-7b2c-4eed-bc87-589910c7118c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:30.969946 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e133eb8-7b2c-4eed-bc87-589910c7118c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:30.971641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e133eb8-7b2c-4eed-bc87-589910c7118c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:30.972121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f22a1-fb47-43bb-913f-c2adcf4992a1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:31.006998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a7261-8c1f-47c6-a205-a94a2bf0e5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:31.007622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a7261-8c1f-47c6-a205-a94a2bf0e5e6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:31.008481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a7261-8c1f-47c6-a205-a94a2bf0e5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:31.008767 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a7261-8c1f-47c6-a205-a94a2bf0e5e6 None admin] PATH_INFO: `/v3/users/4694cfe2faf74f7085cc07187b0f047f` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:31.009907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a7261-8c1f-47c6-a205-a94a2bf0e5e6 None admin] RBAC: Authorizing `identity:get_user(user_id=4694cfe2faf74f7085cc07187b0f047f)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:31.011373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a7261-8c1f-47c6-a205-a94a2bf0e5e6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:31.012945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 37/83] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:30 2022] GET /identity/v3/users/4694cfe2faf74f7085cc07187b0f047f => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:31.013691 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a7261-8c1f-47c6-a205-a94a2bf0e5e6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:31.068347 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 47/84] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:31.069194 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e133eb8-7b2c-4eed-bc87-589910c7118c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:31.080852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:31.085639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:31.116284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a7261-8c1f-47c6-a205-a94a2bf0e5e6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:31.152118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00461086-e537-4184-afb1-d29632c1979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:31.152735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00461086-e537-4184-afb1-d29632c1979d None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:31.153019 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00461086-e537-4184-afb1-d29632c1979d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:31.153298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00461086-e537-4184-afb1-d29632c1979d None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:31.153949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00461086-e537-4184-afb1-d29632c1979d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:31.155415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00461086-e537-4184-afb1-d29632c1979d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:31.168984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e133eb8-7b2c-4eed-bc87-589910c7118c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:31.201068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3ba70-111a-43d1-9acd-168a01a8ce54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:31.201662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3ba70-111a-43d1-9acd-168a01a8ce54 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:31.201936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3ba70-111a-43d1-9acd-168a01a8ce54 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:31.202224 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3ba70-111a-43d1-9acd-168a01a8ce54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:31.263528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 38/85] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:36:31 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:36:31.264328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00461086-e537-4184-afb1-d29632c1979d None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:31.276602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3ba70-111a-43d1-9acd-168a01a8ce54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:31.278815 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3ba70-111a-43d1-9acd-168a01a8ce54 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:31.291193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:31.386286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00461086-e537-4184-afb1-d29632c1979d None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:31.395441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 48/86] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:31.396640 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3ba70-111a-43d1-9acd-168a01a8ce54 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:31.415956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:31.423932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e77f7-c773-4723-8df4-135e3011272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:31.424591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e77f7-c773-4723-8df4-135e3011272f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:31.429279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e77f7-c773-4723-8df4-135e3011272f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:31.429279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e77f7-c773-4723-8df4-135e3011272f None admin] PATH_INFO: `/v3/users/4694cfe2faf74f7085cc07187b0f047f` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:31.429279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e77f7-c773-4723-8df4-135e3011272f None admin] RBAC: Authorizing `identity:get_user(user_id=4694cfe2faf74f7085cc07187b0f047f)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:31.429279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e77f7-c773-4723-8df4-135e3011272f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:31.429551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 39/87] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:31 2022] GET /identity/v3/users/4694cfe2faf74f7085cc07187b0f047f => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:31.430393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e77f7-c773-4723-8df4-135e3011272f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:31.522811 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3ba70-111a-43d1-9acd-168a01a8ce54 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:31.535741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:31.564779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e77f7-c773-4723-8df4-135e3011272f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:31.579276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624122f4-f477-4310-b985-c37203f6a174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:31.580179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624122f4-f477-4310-b985-c37203f6a174 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:31.580654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624122f4-f477-4310-b985-c37203f6a174 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:31.581090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624122f4-f477-4310-b985-c37203f6a174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:31.617053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5274a-98e1-4a15-9ade-e89f1e688546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:31.618359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5274a-98e1-4a15-9ade-e89f1e688546 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:31.618826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5274a-98e1-4a15-9ade-e89f1e688546 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:31.619117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5274a-98e1-4a15-9ade-e89f1e688546 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:31.620284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5274a-98e1-4a15-9ade-e89f1e688546 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:31.621739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5274a-98e1-4a15-9ade-e89f1e688546 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:31.623490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 40/88] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:31 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:31.624186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5274a-98e1-4a15-9ade-e89f1e688546 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:31.679186 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624122f4-f477-4310-b985-c37203f6a174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:31.682207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624122f4-f477-4310-b985-c37203f6a174 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:31.704141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:31.755610 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5274a-98e1-4a15-9ade-e89f1e688546 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:31.791905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bafd6-f4b0-466f-a8cd-bad26b889670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:31.792882 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bafd6-f4b0-466f-a8cd-bad26b889670 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:31.793344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bafd6-f4b0-466f-a8cd-bad26b889670 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:31.793791 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bafd6-f4b0-466f-a8cd-bad26b889670 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:31.795140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bafd6-f4b0-466f-a8cd-bad26b889670 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:31.796819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bafd6-f4b0-466f-a8cd-bad26b889670 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:31.802591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 41/89] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:31 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:31.803741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bafd6-f4b0-466f-a8cd-bad26b889670 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:31.813857 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 49/90] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:31.814736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624122f4-f477-4310-b985-c37203f6a174 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:31.826117 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:31.919479 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bafd6-f4b0-466f-a8cd-bad26b889670 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:31.956814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624122f4-f477-4310-b985-c37203f6a174 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:31.966898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:31.971117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59b40b-b25c-451c-927d-051ce20b393a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:31.971760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59b40b-b25c-451c-927d-051ce20b393a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:31.973600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59b40b-b25c-451c-927d-051ce20b393a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:31.973876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59b40b-b25c-451c-927d-051ce20b393a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:32.001871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9d97a-c7f4-43dd-a633-06578ea11f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:32.002504 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9d97a-c7f4-43dd-a633-06578ea11f7f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:32.002792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9d97a-c7f4-43dd-a633-06578ea11f7f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:32.003071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9d97a-c7f4-43dd-a633-06578ea11f7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:32.003804 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9d97a-c7f4-43dd-a633-06578ea11f7f None admin] RBAC: Adding query filter params (role.id=43667d074eda4143bf3b7f721e83810c, scope.project.id=8b0387feea344c35a5907f20e962c357, user.id=4694cfe2faf74f7085cc07187b0f047f) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:32.004114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9d97a-c7f4-43dd-a633-06578ea11f7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:32.006218 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9d97a-c7f4-43dd-a633-06578ea11f7f None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:32.015248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 50/91] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:31 2022] GET /identity/v3/role_assignments?user.id=4694cfe2faf74f7085cc07187b0f047f&scope.project.id=8b0387feea344c35a5907f20e962c357&role.id=43667d074eda4143bf3b7f721e83810c => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:32.015967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9d97a-c7f4-43dd-a633-06578ea11f7f None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:32.085764 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59b40b-b25c-451c-927d-051ce20b393a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:32.088225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59b40b-b25c-451c-927d-051ce20b393a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:32.202155 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 42/92] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:32.203094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59b40b-b25c-451c-927d-051ce20b393a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:32.229656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:32.238970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:32.294365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9d97a-c7f4-43dd-a633-06578ea11f7f None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:32.334221 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0a9fe-4b1e-4382-9785-e6ab8f3b4ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:32.334911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0a9fe-4b1e-4382-9785-e6ab8f3b4ceb None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:32.335195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0a9fe-4b1e-4382-9785-e6ab8f3b4ceb None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:32.335467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0a9fe-4b1e-4382-9785-e6ab8f3b4ceb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:32.336677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0a9fe-4b1e-4382-9785-e6ab8f3b4ceb None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=8b0387feea344c35a5907f20e962c357, user.id=9947df19bb3b44318e674b5168544c18) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:32.337201 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0a9fe-4b1e-4382-9785-e6ab8f3b4ceb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:32.338323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0a9fe-4b1e-4382-9785-e6ab8f3b4ceb None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:32.342761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59b40b-b25c-451c-927d-051ce20b393a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:32.356180 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 51/93] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:32 2022] GET /identity/v3/role_assignments?user.id=9947df19bb3b44318e674b5168544c18&scope.project.id=8b0387feea344c35a5907f20e962c357&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 259 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:32.357453 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0a9fe-4b1e-4382-9785-e6ab8f3b4ceb None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:32.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd8d01-dcd6-4446-b64b-4474b17976e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:32.430918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd8d01-dcd6-4446-b64b-4474b17976e2 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:32.431432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd8d01-dcd6-4446-b64b-4474b17976e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:32.431906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd8d01-dcd6-4446-b64b-4474b17976e2 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:32.439418 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd8d01-dcd6-4446-b64b-4474b17976e2 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:32.441276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd8d01-dcd6-4446-b64b-4474b17976e2 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:32.443666 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 43/94] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:32 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:32.449994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd8d01-dcd6-4446-b64b-4474b17976e2 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:32.542420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0a9fe-4b1e-4382-9785-e6ab8f3b4ceb None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:32.579187 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd8d01-dcd6-4446-b64b-4474b17976e2 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:32.582816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:32.607350 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8168068-fe6e-44cd-8c41-881bef13b75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:32.607924 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8168068-fe6e-44cd-8c41-881bef13b75d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:32.608204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8168068-fe6e-44cd-8c41-881bef13b75d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:32.608476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8168068-fe6e-44cd-8c41-881bef13b75d None admin] PATH_INFO: `/v3/projects/b68edeee792348298de0934e067273bf` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:32.615333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8168068-fe6e-44cd-8c41-881bef13b75d None admin] RBAC: Authorizing `identity:get_project(project_id=b68edeee792348298de0934e067273bf)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:32.616916 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8168068-fe6e-44cd-8c41-881bef13b75d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:32.619519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 52/95] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:32 2022] GET /identity/v3/projects/b68edeee792348298de0934e067273bf => generated 312 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:32.621364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288f158-15fe-4c9b-8581-2a77b4a7e7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:32.622227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288f158-15fe-4c9b-8581-2a77b4a7e7fe None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:32.622762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288f158-15fe-4c9b-8581-2a77b4a7e7fe None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:32.623251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288f158-15fe-4c9b-8581-2a77b4a7e7fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:32.624207 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288f158-15fe-4c9b-8581-2a77b4a7e7fe None admin] RBAC: Adding query filter params (role.id=96271a2245c144d4b43dea66f18074e8, scope.project.id=8b0387feea344c35a5907f20e962c357, user.id=4694cfe2faf74f7085cc07187b0f047f) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:32.624741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288f158-15fe-4c9b-8581-2a77b4a7e7fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:32.626583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288f158-15fe-4c9b-8581-2a77b4a7e7fe None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:32.627501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8168068-fe6e-44cd-8c41-881bef13b75d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:32.631571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:32.634268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 44/96] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:32 2022] GET /identity/v3/role_assignments?user.id=4694cfe2faf74f7085cc07187b0f047f&scope.project.id=8b0387feea344c35a5907f20e962c357&role.id=96271a2245c144d4b43dea66f18074e8 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:32.780566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8168068-fe6e-44cd-8c41-881bef13b75d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:32.819077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e326a19-b0a4-4e7a-af7a-a14b5cd5b60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:32.819715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e326a19-b0a4-4e7a-af7a-a14b5cd5b60a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:32.820000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e326a19-b0a4-4e7a-af7a-a14b5cd5b60a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:32.820276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e326a19-b0a4-4e7a-af7a-a14b5cd5b60a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:32.821305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e326a19-b0a4-4e7a-af7a-a14b5cd5b60a None admin] RBAC: Adding query filter params (role.id=43667d074eda4143bf3b7f721e83810c, scope.project.id=b68edeee792348298de0934e067273bf, user.id=4694cfe2faf74f7085cc07187b0f047f) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:32.821596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e326a19-b0a4-4e7a-af7a-a14b5cd5b60a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:32.823136 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e326a19-b0a4-4e7a-af7a-a14b5cd5b60a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:32.829489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 53/97] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:32 2022] GET /identity/v3/role_assignments?user.id=4694cfe2faf74f7085cc07187b0f047f&scope.project.id=b68edeee792348298de0934e067273bf&role.id=43667d074eda4143bf3b7f721e83810c => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:34.283485 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f87b43-1b99-408a-a494-4e83637ee576 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:34.284510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f87b43-1b99-408a-a494-4e83637ee576 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:34.285007 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f87b43-1b99-408a-a494-4e83637ee576 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:34.286668 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 45/98] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:34.304078 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51a5ab-7297-49e2-9740-934c54623606 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:34.304533 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51a5ab-7297-49e2-9740-934c54623606 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:34.305238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51a5ab-7297-49e2-9740-934c54623606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:34.329874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-7a51a5ab-7297-49e2-9740-934c54623606 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:34.333536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7a51a5ab-7297-49e2-9740-934c54623606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:34.412779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e63071-779b-4063-903e-e8ae503963c0 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:34.413369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e63071-779b-4063-903e-e8ae503963c0 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:34.413838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e63071-779b-4063-903e-e8ae503963c0 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:34.415776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 46/99] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:34.427653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69bb0d-6ce5-4a86-a225-30460572a3dd None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:34.428199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69bb0d-6ce5-4a86-a225-30460572a3dd None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:34.428661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69bb0d-6ce5-4a86-a225-30460572a3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:34.430165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:34.438475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 54/100] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:34.467512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-0f69bb0d-6ce5-4a86-a225-30460572a3dd None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:34.469079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0f69bb0d-6ce5-4a86-a225-30460572a3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:34.563086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 47/101] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:34.769810 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51a5ab-7297-49e2-9740-934c54623606 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:34.882785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51a5ab-7297-49e2-9740-934c54623606 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:34.935391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ccd0e-52cd-4003-a78e-e55192416aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:34.936402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ccd0e-52cd-4003-a78e-e55192416aaf None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:34.936870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ccd0e-52cd-4003-a78e-e55192416aaf None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:34.937328 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ccd0e-52cd-4003-a78e-e55192416aaf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:34.951326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ccd0e-52cd-4003-a78e-e55192416aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-d18ccd0e-52cd-4003-a78e-e55192416aaf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:34.956334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:34.957167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:34.957948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:34.957948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:34.957948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:34.957948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:34.957948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:34.957948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:34.957948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:34.957948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:34.957948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:34.957948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:34.957948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:36:34.957948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:34.960129 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 55/102] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:34.966598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69bb0d-6ce5-4a86-a225-30460572a3dd None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:35.056658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f95336-e7b2-425b-8671-3ecc9c2d24d5 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:35.056890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f95336-e7b2-425b-8671-3ecc9c2d24d5 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:35.057609 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f95336-e7b2-425b-8671-3ecc9c2d24d5 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:35.059452 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 56/103] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:35.076952 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ded3e-4d06-43f1-93c9-45d8a356276e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:35.077112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ded3e-4d06-43f1-93c9-45d8a356276e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:35.077437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ded3e-4d06-43f1-93c9-45d8a356276e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:35.111555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-2e6ded3e-4d06-43f1-93c9-45d8a356276e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:35.113875 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2e6ded3e-4d06-43f1-93c9-45d8a356276e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:35.136386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:35.145476 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69bb0d-6ce5-4a86-a225-30460572a3dd None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:35.206365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:35.210157 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceab607-f85e-45a0-8d8c-15773f28d0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:35.210819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceab607-f85e-45a0-8d8c-15773f28d0db None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:35.211670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceab607-f85e-45a0-8d8c-15773f28d0db None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:35.211940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceab607-f85e-45a0-8d8c-15773f28d0db None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:35.212611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceab607-f85e-45a0-8d8c-15773f28d0db None admin] RBAC: Adding query filter params (name=reader) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:35.212914 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceab607-f85e-45a0-8d8c-15773f28d0db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:35.214749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceab607-f85e-45a0-8d8c-15773f28d0db None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:35.228533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 48/104] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:35.229409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceab607-f85e-45a0-8d8c-15773f28d0db None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:35.244147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 57/105] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:35.245088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ded3e-4d06-43f1-93c9-45d8a356276e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:35.423510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceab607-f85e-45a0-8d8c-15773f28d0db None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:35.435216 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ded3e-4d06-43f1-93c9-45d8a356276e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:35.480830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce452b-a016-4d33-8b63-d38b039cffde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:35.481525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce452b-a016-4d33-8b63-d38b039cffde None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:35.481813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce452b-a016-4d33-8b63-d38b039cffde None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:35.482092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce452b-a016-4d33-8b63-d38b039cffde None admin] PATH_INFO: `/v3/users/158e69e61f0840ed8ad8b22fe178f423` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:35.496237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91700c91-0ac4-4a71-b6b3-158b24038c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:35.496896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91700c91-0ac4-4a71-b6b3-158b24038c16 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:35.497180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91700c91-0ac4-4a71-b6b3-158b24038c16 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:35.497459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91700c91-0ac4-4a71-b6b3-158b24038c16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:35.498722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce452b-a016-4d33-8b63-d38b039cffde None admin] RBAC: Authorizing `identity:get_user(user_id=158e69e61f0840ed8ad8b22fe178f423)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:35.500176 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce452b-a016-4d33-8b63-d38b039cffde None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:35.502197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 58/106] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:35 2022] GET /identity/v3/users/158e69e61f0840ed8ad8b22fe178f423 => generated 289 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:35.503184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91700c91-0ac4-4a71-b6b3-158b24038c16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:35.504648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91700c91-0ac4-4a71-b6b3-158b24038c16 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:35.506282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 49/107] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:35.507271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce452b-a016-4d33-8b63-d38b039cffde None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:35.512218 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91700c91-0ac4-4a71-b6b3-158b24038c16 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:35.557366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:35.595645 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:35.622640 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce452b-a016-4d33-8b63-d38b039cffde None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:35.671272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7648f-35b1-403e-b471-9425af82caa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:35.673974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7648f-35b1-403e-b471-9425af82caa1 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:35.675492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7648f-35b1-403e-b471-9425af82caa1 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:35.675999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7648f-35b1-403e-b471-9425af82caa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:35.791704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91700c91-0ac4-4a71-b6b3-158b24038c16 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:35.807019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7648f-35b1-403e-b471-9425af82caa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:35.809058 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7648f-35b1-403e-b471-9425af82caa1 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:35.855763 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:35.859984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f576c52-02bb-4adb-a7f7-d76cbe2a9bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:35.860631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f576c52-02bb-4adb-a7f7-d76cbe2a9bee None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:35.860921 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f576c52-02bb-4adb-a7f7-d76cbe2a9bee None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:35.861199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f576c52-02bb-4adb-a7f7-d76cbe2a9bee None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:35.861890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f576c52-02bb-4adb-a7f7-d76cbe2a9bee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:35.863497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f576c52-02bb-4adb-a7f7-d76cbe2a9bee None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:35.928940 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 59/108] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:35.930960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7648f-35b1-403e-b471-9425af82caa1 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:35.994890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 50/109] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:36:35 2022] POST /identity/v3/projects => generated 302 bytes in 483 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:36:35.995886 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f576c52-02bb-4adb-a7f7-d76cbe2a9bee None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:36.044673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7648f-35b1-403e-b471-9425af82caa1 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:36.083756 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bb3b8-4a33-40aa-9651-710e290dc6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:36.084414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bb3b8-4a33-40aa-9651-710e290dc6af None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:36.084695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bb3b8-4a33-40aa-9651-710e290dc6af None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:36.084969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bb3b8-4a33-40aa-9651-710e290dc6af None admin] PATH_INFO: `/v3/roles/member` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:36.095313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:36.098313 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bb3b8-4a33-40aa-9651-710e290dc6af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-ea3bb3b8-4a33-40aa-9651-710e290dc6af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:36.103713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:36.105759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 60/110] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:36.106618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd7452-3d5e-46b3-b1b4-66cd3bc7a257 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:36.107333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd7452-3d5e-46b3-b1b4-66cd3bc7a257 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:36.107333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd7452-3d5e-46b3-b1b4-66cd3bc7a257 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:36.108162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 61/111] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:36.108938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0da0e5-0455-4040-b1c3-e608322d0f3a None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:36.109219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0da0e5-0455-4040-b1c3-e608322d0f3a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:36.109767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0da0e5-0455-4040-b1c3-e608322d0f3a None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:36.115535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 62/112] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:36.115535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcab7a-893d-4f19-b2a2-3b4ab2ba82c4 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:36.115535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcab7a-893d-4f19-b2a2-3b4ab2ba82c4 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:36.115535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcab7a-893d-4f19-b2a2-3b4ab2ba82c4 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:36.115535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 63/113] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:36.119510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dcab7a-893d-4f19-b2a2-3b4ab2ba82c4 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:36.124713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:36.144593 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f576c52-02bb-4adb-a7f7-d76cbe2a9bee None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:36.187114 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab2ce2-5bc8-4e37-b0c6-62a86a75927f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:36.187690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab2ce2-5bc8-4e37-b0c6-62a86a75927f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:36.188476 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab2ce2-5bc8-4e37-b0c6-62a86a75927f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:36.188476 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab2ce2-5bc8-4e37-b0c6-62a86a75927f None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:36.189378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab2ce2-5bc8-4e37-b0c6-62a86a75927f None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:36.191187 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab2ce2-5bc8-4e37-b0c6-62a86a75927f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:36.192565 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 51/114] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:35 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:36.193846 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bed74a-c505-4d9b-939d-ee74abee71f7 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:36.194141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bed74a-c505-4d9b-939d-ee74abee71f7 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:36.194420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bed74a-c505-4d9b-939d-ee74abee71f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:36.215728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-04bed74a-c505-4d9b-939d-ee74abee71f7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:36.216928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-04bed74a-c505-4d9b-939d-ee74abee71f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:36.241581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dcab7a-893d-4f19-b2a2-3b4ab2ba82c4 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:36.280355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f582e4b-ba68-4376-b7c0-c59671aa0fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:36.280962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f582e4b-ba68-4376-b7c0-c59671aa0fb6 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:36.281245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f582e4b-ba68-4376-b7c0-c59671aa0fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:36.281520 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f582e4b-ba68-4376-b7c0-c59671aa0fb6 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:36.282173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f582e4b-ba68-4376-b7c0-c59671aa0fb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:36.282471 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f582e4b-ba68-4376-b7c0-c59671aa0fb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:36.283860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f582e4b-ba68-4376-b7c0-c59671aa0fb6 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:36.296353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 64/115] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:36.296353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e5540-a7a2-4f1b-97a4-b1532269409c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:36.296353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e5540-a7a2-4f1b-97a4-b1532269409c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:36.296353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e5540-a7a2-4f1b-97a4-b1532269409c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:36.300589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 52/116] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:36.311854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53260685-6f8c-4c28-8cd7-bda73c6e2f7a None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:36.312145 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53260685-6f8c-4c28-8cd7-bda73c6e2f7a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:36.312414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53260685-6f8c-4c28-8cd7-bda73c6e2f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:36.313034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-531e5540-a7a2-4f1b-97a4-b1532269409c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:36.314233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-531e5540-a7a2-4f1b-97a4-b1532269409c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:36.338531 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-53260685-6f8c-4c28-8cd7-bda73c6e2f7a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:36.339826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-53260685-6f8c-4c28-8cd7-bda73c6e2f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:36.395514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:36.405145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 65/117] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:36.407385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e5540-a7a2-4f1b-97a4-b1532269409c None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:36.433182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 53/118] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:36.435318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53260685-6f8c-4c28-8cd7-bda73c6e2f7a None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:36.441862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:36.544121 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e5540-a7a2-4f1b-97a4-b1532269409c None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:36.578792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dbee4-b028-4f17-9fff-ba2104def07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:36.579396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dbee4-b028-4f17-9fff-ba2104def07e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:36.579672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dbee4-b028-4f17-9fff-ba2104def07e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:36.579940 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dbee4-b028-4f17-9fff-ba2104def07e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:36.583824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dbee4-b028-4f17-9fff-ba2104def07e None admin] RBAC: Adding query filter params (role.id=ca39d64092424a34aaaf4818ca16e988, scope.project.id=8b0387feea344c35a5907f20e962c357, user.id=158e69e61f0840ed8ad8b22fe178f423) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:36.583824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dbee4-b028-4f17-9fff-ba2104def07e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:36.583824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dbee4-b028-4f17-9fff-ba2104def07e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:36.588611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 66/119] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:36 2022] GET /identity/v3/role_assignments?user.id=158e69e61f0840ed8ad8b22fe178f423&scope.project.id=8b0387feea344c35a5907f20e962c357&role.id=ca39d64092424a34aaaf4818ca16e988 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:36.615930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53260685-6f8c-4c28-8cd7-bda73c6e2f7a None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:36.672476 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af52e61-f795-4de9-94c6-fec486e6a3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:36.673466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af52e61-f795-4de9-94c6-fec486e6a3ac None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:36.673940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af52e61-f795-4de9-94c6-fec486e6a3ac None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:36.674404 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af52e61-f795-4de9-94c6-fec486e6a3ac None admin] PATH_INFO: `/v3/users/4694cfe2faf74f7085cc07187b0f047f` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:36.675836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af52e61-f795-4de9-94c6-fec486e6a3ac None admin] RBAC: Authorizing `identity:get_user(user_id=4694cfe2faf74f7085cc07187b0f047f)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:36.677472 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af52e61-f795-4de9-94c6-fec486e6a3ac None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:36.679564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 54/120] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:36 2022] GET /identity/v3/users/4694cfe2faf74f7085cc07187b0f047f => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:36.687804 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dbee4-b028-4f17-9fff-ba2104def07e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:36.730195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af52e61-f795-4de9-94c6-fec486e6a3ac None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:36.813129 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:36.871132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:36.889691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dbee4-b028-4f17-9fff-ba2104def07e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:36.902756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af52e61-f795-4de9-94c6-fec486e6a3ac None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:36.943867 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52b032-474c-4ff1-9dcb-34705b2548b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:36.944795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52b032-474c-4ff1-9dcb-34705b2548b9 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:36.945932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52b032-474c-4ff1-9dcb-34705b2548b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:36.946350 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52b032-474c-4ff1-9dcb-34705b2548b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:36.947177 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c89a9-70b5-41d7-a5bb-874caad4b13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:36.947814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c89a9-70b5-41d7-a5bb-874caad4b13e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:36.948104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c89a9-70b5-41d7-a5bb-874caad4b13e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:36.948381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c89a9-70b5-41d7-a5bb-874caad4b13e None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:36.949683 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c89a9-70b5-41d7-a5bb-874caad4b13e None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:36.951667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c89a9-70b5-41d7-a5bb-874caad4b13e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:36.953770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 67/121] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:36 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:36.954380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c89a9-70b5-41d7-a5bb-874caad4b13e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:36.963430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52b032-474c-4ff1-9dcb-34705b2548b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-5b52b032-474c-4ff1-9dcb-34705b2548b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:36.968490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:36.969335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:36.970099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:36.970099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:36.970099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:36.970099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:36.970099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:36.970099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:36.970099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:36.970099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:36.970099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:36.970099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:36.970099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:36.970099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:36:36.972327 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 55/122] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:36:36.977786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52b032-474c-4ff1-9dcb-34705b2548b9 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:37.082134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52b032-474c-4ff1-9dcb-34705b2548b9 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:37.090424 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c89a9-70b5-41d7-a5bb-874caad4b13e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:37.116178 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:37.125173 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7999392-6bcd-4c9f-a1b5-521bb8f5a00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:37.125865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7999392-6bcd-4c9f-a1b5-521bb8f5a00e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:37.130192 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7999392-6bcd-4c9f-a1b5-521bb8f5a00e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:37.130192 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7999392-6bcd-4c9f-a1b5-521bb8f5a00e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:37.133229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:37.137548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7999392-6bcd-4c9f-a1b5-521bb8f5a00e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-c7999392-6bcd-4c9f-a1b5-521bb8f5a00e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:37.142370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:37.143270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:37.144046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:37.144046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:37.144046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:37.144046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:37.144046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:37.144046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:37.144046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:37.144046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:37.144046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:37.144046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:37.144046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:37.144046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:36:37.144046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 56/123] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:36:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:37.149479 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7999392-6bcd-4c9f-a1b5-521bb8f5a00e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:37.149602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fa64b-13ae-49dd-886d-f13e46c9dac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:37.149602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fa64b-13ae-49dd-886d-f13e46c9dac2 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:37.149602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fa64b-13ae-49dd-886d-f13e46c9dac2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:37.149818 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fa64b-13ae-49dd-886d-f13e46c9dac2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:37.165539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fa64b-13ae-49dd-886d-f13e46c9dac2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-693fa64b-13ae-49dd-886d-f13e46c9dac2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:37.173968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:37.174868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:37.175663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:37.175663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:37.175663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:37.175663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:37.175663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:37.175663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:37.175663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:37.175663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:37.175663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:37.175663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:37.175663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:36:37.175663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:37.180291 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 68/124] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:36:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:37.181740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fa64b-13ae-49dd-886d-f13e46c9dac2 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:37.277575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7999392-6bcd-4c9f-a1b5-521bb8f5a00e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:37.315625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07740659-c1fb-4d72-b05d-71551b2bff5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:37.316275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07740659-c1fb-4d72-b05d-71551b2bff5f None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:37.316556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07740659-c1fb-4d72-b05d-71551b2bff5f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:37.316833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07740659-c1fb-4d72-b05d-71551b2bff5f None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357/users/4694cfe2faf74f7085cc07187b0f047f/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:37.318790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07740659-c1fb-4d72-b05d-71551b2bff5f None admin] RBAC: Authorizing `identity:create_grant(project_id=8b0387feea344c35a5907f20e962c357, user_id=4694cfe2faf74f7085cc07187b0f047f, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:37.320320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07740659-c1fb-4d72-b05d-71551b2bff5f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:37.325113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fa64b-13ae-49dd-886d-f13e46c9dac2 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:37.361259 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1aa82-ac6a-49f9-901c-ba9fb145d14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:37.361850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1aa82-ac6a-49f9-901c-ba9fb145d14f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:37.362138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1aa82-ac6a-49f9-901c-ba9fb145d14f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:37.362414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1aa82-ac6a-49f9-901c-ba9fb145d14f None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:37.363160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1aa82-ac6a-49f9-901c-ba9fb145d14f None admin] RBAC: Adding query filter params (name=member) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:37.363467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1aa82-ac6a-49f9-901c-ba9fb145d14f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:37.364893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1aa82-ac6a-49f9-901c-ba9fb145d14f None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:37.378998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 69/125] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:37.378998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1aa82-ac6a-49f9-901c-ba9fb145d14f None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:37.399600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 57/126] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:36:37 2022] PUT /identity/v3/projects/8b0387feea344c35a5907f20e962c357/users/4694cfe2faf74f7085cc07187b0f047f/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:36:37.400773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07740659-c1fb-4d72-b05d-71551b2bff5f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:37.464899 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:37.474861 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:37.529863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07740659-c1fb-4d72-b05d-71551b2bff5f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:37.570991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a442a-c6d8-48c3-a0c4-c37c2d227096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:37.572022 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a442a-c6d8-48c3-a0c4-c37c2d227096 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:37.572477 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a442a-c6d8-48c3-a0c4-c37c2d227096 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:37.572918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a442a-c6d8-48c3-a0c4-c37c2d227096 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:37.573792 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a442a-c6d8-48c3-a0c4-c37c2d227096 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:37.574268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a442a-c6d8-48c3-a0c4-c37c2d227096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:37.576291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a442a-c6d8-48c3-a0c4-c37c2d227096 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:37.586672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 58/127] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:36:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:37.587823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a442a-c6d8-48c3-a0c4-c37c2d227096 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:37.600996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1aa82-ac6a-49f9-901c-ba9fb145d14f None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:37.708343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b30e5b-0ecc-40d6-81f0-dbbc1ba7fe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:37.708928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b30e5b-0ecc-40d6-81f0-dbbc1ba7fe1a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:37.709212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b30e5b-0ecc-40d6-81f0-dbbc1ba7fe1a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:37.709602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b30e5b-0ecc-40d6-81f0-dbbc1ba7fe1a None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:37.710286 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b30e5b-0ecc-40d6-81f0-dbbc1ba7fe1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:37.710624 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b30e5b-0ecc-40d6-81f0-dbbc1ba7fe1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:37.712499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a442a-c6d8-48c3-a0c4-c37c2d227096 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:37.712610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b30e5b-0ecc-40d6-81f0-dbbc1ba7fe1a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:37.734509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:37.744704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 70/128] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:37.744704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b30e5b-0ecc-40d6-81f0-dbbc1ba7fe1a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:37.775633 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b41773-1fa8-44ad-8622-d6c17e9b86c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:37.776286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b41773-1fa8-44ad-8622-d6c17e9b86c7 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:37.776995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b41773-1fa8-44ad-8622-d6c17e9b86c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:37.777282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b41773-1fa8-44ad-8622-d6c17e9b86c7 None admin] PATH_INFO: `/v3/users/4694cfe2faf74f7085cc07187b0f047f` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:37.778732 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b41773-1fa8-44ad-8622-d6c17e9b86c7 None admin] RBAC: Authorizing `identity:get_user(user_id=4694cfe2faf74f7085cc07187b0f047f)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:37.780824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b41773-1fa8-44ad-8622-d6c17e9b86c7 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:37.782857 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 59/129] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:37 2022] GET /identity/v3/users/4694cfe2faf74f7085cc07187b0f047f => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:37.784597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b41773-1fa8-44ad-8622-d6c17e9b86c7 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:37.870705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b30e5b-0ecc-40d6-81f0-dbbc1ba7fe1a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:37.884100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:37.904626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b41773-1fa8-44ad-8622-d6c17e9b86c7 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:37.913327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbed8c0-a52e-41f8-aa79-88eb150f08d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:37.913988 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbed8c0-a52e-41f8-aa79-88eb150f08d8 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:37.914888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbed8c0-a52e-41f8-aa79-88eb150f08d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:37.915168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbed8c0-a52e-41f8-aa79-88eb150f08d8 None admin] PATH_INFO: `/v3/users/4694cfe2faf74f7085cc07187b0f047f` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:37.919187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbed8c0-a52e-41f8-aa79-88eb150f08d8 None admin] RBAC: Authorizing `identity:get_user(user_id=4694cfe2faf74f7085cc07187b0f047f)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:37.920711 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbed8c0-a52e-41f8-aa79-88eb150f08d8 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:37.922673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 71/130] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:37 2022] GET /identity/v3/users/4694cfe2faf74f7085cc07187b0f047f => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:37.923460 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbed8c0-a52e-41f8-aa79-88eb150f08d8 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:37.941636 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1f789-07e1-4918-8700-03bbfebd2333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:37.942280 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1f789-07e1-4918-8700-03bbfebd2333 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:37.942594 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1f789-07e1-4918-8700-03bbfebd2333 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:37.942876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1f789-07e1-4918-8700-03bbfebd2333 None admin] PATH_INFO: `/v3/users/9947df19bb3b44318e674b5168544c18` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:37.944015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1f789-07e1-4918-8700-03bbfebd2333 None admin] RBAC: Authorizing `identity:get_user(user_id=9947df19bb3b44318e674b5168544c18)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:37.945562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1f789-07e1-4918-8700-03bbfebd2333 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:37.947187 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 60/131] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:37 2022] GET /identity/v3/users/9947df19bb3b44318e674b5168544c18 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:37.947962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1f789-07e1-4918-8700-03bbfebd2333 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:38.001352 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:38.066305 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1f789-07e1-4918-8700-03bbfebd2333 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:38.116130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbed8c0-a52e-41f8-aa79-88eb150f08d8 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:38.131249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d49d75-43de-480b-9e75-47e102b12083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:38.131895 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d49d75-43de-480b-9e75-47e102b12083 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:38.132177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d49d75-43de-480b-9e75-47e102b12083 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:38.132451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d49d75-43de-480b-9e75-47e102b12083 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:38.134309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d49d75-43de-480b-9e75-47e102b12083 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:38.135789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d49d75-43de-480b-9e75-47e102b12083 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:38.140193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 61/132] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:37 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:38.141854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d49d75-43de-480b-9e75-47e102b12083 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:38.184295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444a9bf-2575-4b32-84cb-9f15b74cfcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:38.184934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444a9bf-2575-4b32-84cb-9f15b74cfcd9 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:38.185211 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444a9bf-2575-4b32-84cb-9f15b74cfcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:38.185478 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444a9bf-2575-4b32-84cb-9f15b74cfcd9 None admin] PATH_INFO: `/v3/projects/b68edeee792348298de0934e067273bf` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:38.186639 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444a9bf-2575-4b32-84cb-9f15b74cfcd9 None admin] RBAC: Authorizing `identity:get_project(project_id=b68edeee792348298de0934e067273bf)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:38.188066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444a9bf-2575-4b32-84cb-9f15b74cfcd9 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:38.189689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 72/133] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:37 2022] GET /identity/v3/projects/b68edeee792348298de0934e067273bf => generated 312 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:38.190435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444a9bf-2575-4b32-84cb-9f15b74cfcd9 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:38.303020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:38.329835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d49d75-43de-480b-9e75-47e102b12083 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:38.362854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:38.388652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444a9bf-2575-4b32-84cb-9f15b74cfcd9 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:38.402435 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ed428-1355-4e38-9981-65fa775fcbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:38.403441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ed428-1355-4e38-9981-65fa775fcbcc None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:38.403937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ed428-1355-4e38-9981-65fa775fcbcc None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:38.404451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ed428-1355-4e38-9981-65fa775fcbcc None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:38.405594 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ed428-1355-4e38-9981-65fa775fcbcc None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:38.407140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ed428-1355-4e38-9981-65fa775fcbcc None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:38.408933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 62/134] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:38 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:38.409659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ed428-1355-4e38-9981-65fa775fcbcc None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:38.425996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90eb4b-f351-4c91-a8f2-e26636cd4e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:38.426996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90eb4b-f351-4c91-a8f2-e26636cd4e79 None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:38.427282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90eb4b-f351-4c91-a8f2-e26636cd4e79 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:38.427824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90eb4b-f351-4c91-a8f2-e26636cd4e79 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357/users/4694cfe2faf74f7085cc07187b0f047f/roles/96271a2245c144d4b43dea66f18074e8` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:38.430603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90eb4b-f351-4c91-a8f2-e26636cd4e79 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b0387feea344c35a5907f20e962c357, user_id=4694cfe2faf74f7085cc07187b0f047f, role_id=96271a2245c144d4b43dea66f18074e8)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:38.432381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90eb4b-f351-4c91-a8f2-e26636cd4e79 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:38.519749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ed428-1355-4e38-9981-65fa775fcbcc None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:38.536401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 73/135] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:36:38 2022] PUT /identity/v3/projects/8b0387feea344c35a5907f20e962c357/users/4694cfe2faf74f7085cc07187b0f047f/roles/96271a2245c144d4b43dea66f18074e8 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:36:38.545056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f359baa-50c8-4544-ab05-056919852b97 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:38.545647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f359baa-50c8-4544-ab05-056919852b97 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:38.546130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f359baa-50c8-4544-ab05-056919852b97 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:38.548334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 74/136] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:38.549502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f359baa-50c8-4544-ab05-056919852b97 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:38.575963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfc367-a29e-4046-bb1c-7f25ee556e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:38.576742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfc367-a29e-4046-bb1c-7f25ee556e97 None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:38.577151 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfc367-a29e-4046-bb1c-7f25ee556e97 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:38.577502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfc367-a29e-4046-bb1c-7f25ee556e97 None admin] PATH_INFO: `/v3/projects/b68edeee792348298de0934e067273bf/users/4694cfe2faf74f7085cc07187b0f047f/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:38.579743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfc367-a29e-4046-bb1c-7f25ee556e97 None admin] RBAC: Authorizing `identity:create_grant(project_id=b68edeee792348298de0934e067273bf, user_id=4694cfe2faf74f7085cc07187b0f047f, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:38.581489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfc367-a29e-4046-bb1c-7f25ee556e97 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:38.654337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 63/137] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:36:38 2022] PUT /identity/v3/projects/b68edeee792348298de0934e067273bf/users/4694cfe2faf74f7085cc07187b0f047f/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:36:38.655525 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68f152-c25f-4b94-a0c5-b6cb8826b496 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:38.655861 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68f152-c25f-4b94-a0c5-b6cb8826b496 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:38.656444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68f152-c25f-4b94-a0c5-b6cb8826b496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:38.702003 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ce68f152-c25f-4b94-a0c5-b6cb8826b496 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:38.706734 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ce68f152-c25f-4b94-a0c5-b6cb8826b496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:38.794517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f359baa-50c8-4544-ab05-056919852b97 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:38.806702 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:38.807584 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:38.837385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 64/138] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:38.844119 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441de34-860d-4857-a0c1-fa480db7877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:38.845215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441de34-860d-4857-a0c1-fa480db7877a None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:38.845697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441de34-860d-4857-a0c1-fa480db7877a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:38.846182 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441de34-860d-4857-a0c1-fa480db7877a None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357/users/9947df19bb3b44318e674b5168544c18/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:38.851383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441de34-860d-4857-a0c1-fa480db7877a None admin] RBAC: Authorizing `identity:create_grant(project_id=8b0387feea344c35a5907f20e962c357, user_id=9947df19bb3b44318e674b5168544c18, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:38.853780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441de34-860d-4857-a0c1-fa480db7877a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:38.967393 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 75/139] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:36:38 2022] PUT /identity/v3/projects/8b0387feea344c35a5907f20e962c357/users/9947df19bb3b44318e674b5168544c18/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:36:39.164940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68f152-c25f-4b94-a0c5-b6cb8826b496 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:39.257020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29412756-c293-41b5-ae47-06098255048b None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:39.257333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29412756-c293-41b5-ae47-06098255048b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:39.257578 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29412756-c293-41b5-ae47-06098255048b None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:39.259888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 76/140] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:39.276223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c2dfe-1a1d-4f2e-a60d-efb97b32af18 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:39.276407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c2dfe-1a1d-4f2e-a60d-efb97b32af18 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:39.276627 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c2dfe-1a1d-4f2e-a60d-efb97b32af18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:39.296789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68f152-c25f-4b94-a0c5-b6cb8826b496 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:39.315567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-ce2c2dfe-1a1d-4f2e-a60d-efb97b32af18 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:39.316898 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ce2c2dfe-1a1d-4f2e-a60d-efb97b32af18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:39.343653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475700af-8fd5-42e5-98d5-24ffa698a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:39.344317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475700af-8fd5-42e5-98d5-24ffa698a16d None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:39.344604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475700af-8fd5-42e5-98d5-24ffa698a16d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:39.344885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475700af-8fd5-42e5-98d5-24ffa698a16d None admin] PATH_INFO: `/v3/domains/default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:39.346121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475700af-8fd5-42e5-98d5-24ffa698a16d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:39.347616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475700af-8fd5-42e5-98d5-24ffa698a16d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:39.349372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 65/141] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:39.355187 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475700af-8fd5-42e5-98d5-24ffa698a16d None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:39.426616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:39.447054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 77/142] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:39.519397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475700af-8fd5-42e5-98d5-24ffa698a16d None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:39.563366 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc926e0-00a8-4745-b634-09144b2799c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:39.564129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc926e0-00a8-4745-b634-09144b2799c3 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:39.564328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc926e0-00a8-4745-b634-09144b2799c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:39.564570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc926e0-00a8-4745-b634-09144b2799c3 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:39.565242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc926e0-00a8-4745-b634-09144b2799c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:39.566702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc926e0-00a8-4745-b634-09144b2799c3 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:39.722525 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 66/143] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:36:39 2022] POST /identity/v3/users => generated 283 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:36:39.968197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c2dfe-1a1d-4f2e-a60d-efb97b32af18 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:40.087876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:40.091389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c2dfe-1a1d-4f2e-a60d-efb97b32af18 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:40.132183 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28fe44-24d0-4ed5-a4d3-ff36a5a0330f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:40.133504 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28fe44-24d0-4ed5-a4d3-ff36a5a0330f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:40.133919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28fe44-24d0-4ed5-a4d3-ff36a5a0330f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:40.134201 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28fe44-24d0-4ed5-a4d3-ff36a5a0330f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:40.151382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28fe44-24d0-4ed5-a4d3-ff36a5a0330f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-eb28fe44-24d0-4ed5-a4d3-ff36a5a0330f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:40.156365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:40.157497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:40.158555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:40.158555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:40.158555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:40.158555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:40.158555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:40.158555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:40.158555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:40.158555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:40.158555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:40.158555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:40.158555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:36:40.158555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:40.158555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 78/144] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:40.163674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc926e0-00a8-4745-b634-09144b2799c3 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:40.324469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:40.328464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc926e0-00a8-4745-b634-09144b2799c3 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:40.373339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8276e4-3a94-47fd-a132-af610cce180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:40.373883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8276e4-3a94-47fd-a132-af610cce180c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:40.374164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8276e4-3a94-47fd-a132-af610cce180c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:40.374469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8276e4-3a94-47fd-a132-af610cce180c None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:40.375192 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8276e4-3a94-47fd-a132-af610cce180c None admin] RBAC: Adding query filter params (name=reader) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:40.375496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8276e4-3a94-47fd-a132-af610cce180c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:40.376869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8276e4-3a94-47fd-a132-af610cce180c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:40.392316 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 67/145] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:40.398247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28fe44-24d0-4ed5-a4d3-ff36a5a0330f None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:40.544183 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28fe44-24d0-4ed5-a4d3-ff36a5a0330f None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:40.591047 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673c83f-436b-4537-be7c-dfebc0806777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:40.591699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673c83f-436b-4537-be7c-dfebc0806777 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:40.591980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673c83f-436b-4537-be7c-dfebc0806777 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:40.592258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673c83f-436b-4537-be7c-dfebc0806777 None admin] PATH_INFO: `/v3/users/158e69e61f0840ed8ad8b22fe178f423` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:40.599249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673c83f-436b-4537-be7c-dfebc0806777 None admin] RBAC: Authorizing `identity:get_user(user_id=158e69e61f0840ed8ad8b22fe178f423)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:40.600793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673c83f-436b-4537-be7c-dfebc0806777 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:40.602820 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 79/146] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:40 2022] GET /identity/v3/users/158e69e61f0840ed8ad8b22fe178f423 => generated 289 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:40.608394 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8276e4-3a94-47fd-a132-af610cce180c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:40.703737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46ed8a-dc39-4761-b0fb-f33d616ddd20 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:40.703939 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46ed8a-dc39-4761-b0fb-f33d616ddd20 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:40.704217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46ed8a-dc39-4761-b0fb-f33d616ddd20 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:40.705854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 80/147] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:40.714223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08eedf-2535-4e25-aaf2-efe21a39b37f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:40.714803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08eedf-2535-4e25-aaf2-efe21a39b37f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:40.715543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08eedf-2535-4e25-aaf2-efe21a39b37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:40.740064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-1a08eedf-2535-4e25-aaf2-efe21a39b37f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:40.742121 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1a08eedf-2535-4e25-aaf2-efe21a39b37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:40.757650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8276e4-3a94-47fd-a132-af610cce180c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:40.767959 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:40.815561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4a61d-8b6b-44f6-b182-306d78345e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:40.816577 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4a61d-8b6b-44f6-b182-306d78345e1a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:40.817067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4a61d-8b6b-44f6-b182-306d78345e1a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:40.818204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4a61d-8b6b-44f6-b182-306d78345e1a None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:40.819835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4a61d-8b6b-44f6-b182-306d78345e1a None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:40.823244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4a61d-8b6b-44f6-b182-306d78345e1a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:40.826339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 68/148] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:40 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:40.832419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4a61d-8b6b-44f6-b182-306d78345e1a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:40.846384 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 81/149] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:40.874042 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:40.954180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4a61d-8b6b-44f6-b182-306d78345e1a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:40.994320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acc854-4378-4be6-af19-192b53f3acb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:40.995337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acc854-4378-4be6-af19-192b53f3acb0 None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:40.995832 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acc854-4378-4be6-af19-192b53f3acb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:40.996295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acc854-4378-4be6-af19-192b53f3acb0 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357/users/158e69e61f0840ed8ad8b22fe178f423/roles/ca39d64092424a34aaaf4818ca16e988` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:40.998435 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acc854-4378-4be6-af19-192b53f3acb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b0387feea344c35a5907f20e962c357, user_id=158e69e61f0840ed8ad8b22fe178f423, role_id=ca39d64092424a34aaaf4818ca16e988)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:41.000104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acc854-4378-4be6-af19-192b53f3acb0 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:41.069707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 69/150] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:36:40 2022] PUT /identity/v3/projects/8b0387feea344c35a5907f20e962c357/users/158e69e61f0840ed8ad8b22fe178f423/roles/ca39d64092424a34aaaf4818ca16e988 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:36:41.440666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08eedf-2535-4e25-aaf2-efe21a39b37f None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:41.550681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08eedf-2535-4e25-aaf2-efe21a39b37f None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:41.588806 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969787c-352f-43d8-b073-064389ba36f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:41.589441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969787c-352f-43d8-b073-064389ba36f7 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:41.589716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969787c-352f-43d8-b073-064389ba36f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:41.589983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969787c-352f-43d8-b073-064389ba36f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:41.601755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969787c-352f-43d8-b073-064389ba36f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-e969787c-352f-43d8-b073-064389ba36f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:41.606717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:41.607580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:41.608910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:41.608910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:41.608910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:41.608910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:41.608910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:41.608910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:41.608910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:41.608910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:41.608910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:41.608910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:41.608910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:41.608910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:41.608910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 82/151] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:41.613902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acc854-4378-4be6-af19-192b53f3acb0 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:41.696316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1bfb4-85e2-4267-ad33-b0d34d57bb5b None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:41.696598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1bfb4-85e2-4267-ad33-b0d34d57bb5b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:41.696776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1bfb4-85e2-4267-ad33-b0d34d57bb5b None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:41.698438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 83/152] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:36:41.703880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a83a3-a6ca-48ed-908e-bae9274bb053 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:41.704146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a83a3-a6ca-48ed-908e-bae9274bb053 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:41.704577 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a83a3-a6ca-48ed-908e-bae9274bb053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:41.741515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-037a83a3-a6ca-48ed-908e-bae9274bb053 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:41.744193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-037a83a3-a6ca-48ed-908e-bae9274bb053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:41.785385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:41.806889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:41.817409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acc854-4378-4be6-af19-192b53f3acb0 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:41.841437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 84/153] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:41.899206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae87aa4-e085-49e9-9074-2bf08d160399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:41.899831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae87aa4-e085-49e9-9074-2bf08d160399 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:41.900112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae87aa4-e085-49e9-9074-2bf08d160399 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:41.900387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae87aa4-e085-49e9-9074-2bf08d160399 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:41.901062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae87aa4-e085-49e9-9074-2bf08d160399 None admin] RBAC: Adding query filter params (name=member) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:41.901364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae87aa4-e085-49e9-9074-2bf08d160399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:41.902780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae87aa4-e085-49e9-9074-2bf08d160399 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:41.912561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 70/154] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:41.918880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a83a3-a6ca-48ed-908e-bae9274bb053 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:42.078060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a83a3-a6ca-48ed-908e-bae9274bb053 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:42.125589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ab23e-5d56-4145-b2a6-0f3dc7fa047f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:42.126239 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ab23e-5d56-4145-b2a6-0f3dc7fa047f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:42.126848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ab23e-5d56-4145-b2a6-0f3dc7fa047f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:42.127124 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ab23e-5d56-4145-b2a6-0f3dc7fa047f None admin] PATH_INFO: `/v3/users/4694cfe2faf74f7085cc07187b0f047f` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:42.128282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ab23e-5d56-4145-b2a6-0f3dc7fa047f None admin] RBAC: Authorizing `identity:get_user(user_id=4694cfe2faf74f7085cc07187b0f047f)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:42.129768 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ab23e-5d56-4145-b2a6-0f3dc7fa047f None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:42.131374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 85/155] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:41 2022] GET /identity/v3/users/4694cfe2faf74f7085cc07187b0f047f => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:42.139615 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae87aa4-e085-49e9-9074-2bf08d160399 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:42.307789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae87aa4-e085-49e9-9074-2bf08d160399 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:42.360026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ea29f-cf66-4750-91bd-da2571d8c21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:42.360562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ea29f-cf66-4750-91bd-da2571d8c21f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:42.360843 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ea29f-cf66-4750-91bd-da2571d8c21f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:42.361118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ea29f-cf66-4750-91bd-da2571d8c21f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:42.394150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:42.449618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ea29f-cf66-4750-91bd-da2571d8c21f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:42.451148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ea29f-cf66-4750-91bd-da2571d8c21f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:42.510878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b682e2-b232-414e-be90-9e25f23382a9 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:42.511388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b682e2-b232-414e-be90-9e25f23382a9 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:42.511388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b682e2-b232-414e-be90-9e25f23382a9 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:42.513024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 86/156] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:42.517965 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b682e2-b232-414e-be90-9e25f23382a9 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:42.546840 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:42.654859 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 71/157] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:42.656092 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36051da-cf20-4a03-b74f-a584e4307a86 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:42.656381 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36051da-cf20-4a03-b74f-a584e4307a86 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:42.656656 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36051da-cf20-4a03-b74f-a584e4307a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:42.663216 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b682e2-b232-414e-be90-9e25f23382a9 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:42.701489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-a36051da-cf20-4a03-b74f-a584e4307a86 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:42.702703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a36051da-cf20-4a03-b74f-a584e4307a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:42.725435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b958e7-1c01-4918-8712-010d2f0f2875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:42.726515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b958e7-1c01-4918-8712-010d2f0f2875 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:42.727581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b958e7-1c01-4918-8712-010d2f0f2875 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:42.728028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b958e7-1c01-4918-8712-010d2f0f2875 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:42.751548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b958e7-1c01-4918-8712-010d2f0f2875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-99b958e7-1c01-4918-8712-010d2f0f2875 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:42.757525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:42.758981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:42.759775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:42.759775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:42.759775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:42.759775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:42.759775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:42.759775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:42.759775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:42.759775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:42.759775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:42.759775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:42.759775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:36:42.759775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:42.762649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 87/158] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:36:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:36:42.763684 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:42.772074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8aee52-da0d-4176-8782-5e6f0ac48af7 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:42.772597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8aee52-da0d-4176-8782-5e6f0ac48af7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:42.773014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8aee52-da0d-4176-8782-5e6f0ac48af7 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:42.774906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 88/159] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:42 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:42.775849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8aee52-da0d-4176-8782-5e6f0ac48af7 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:42.784996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 72/160] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:42.791150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36051da-cf20-4a03-b74f-a584e4307a86 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:42.890468 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:42.917674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8aee52-da0d-4176-8782-5e6f0ac48af7 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:42.954352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6837e9-6b72-4bd4-a215-5306232eae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:42.955175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6837e9-6b72-4bd4-a215-5306232eae90 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:42.955585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6837e9-6b72-4bd4-a215-5306232eae90 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:42.955971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6837e9-6b72-4bd4-a215-5306232eae90 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:42.957299 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6837e9-6b72-4bd4-a215-5306232eae90 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:42.958889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6837e9-6b72-4bd4-a215-5306232eae90 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:42.960680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 89/161] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:42 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:42.962310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd3b3-ab16-473e-b1c1-8aee8d6a4cb4 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:42.962745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd3b3-ab16-473e-b1c1-8aee8d6a4cb4 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:42.963119 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd3b3-ab16-473e-b1c1-8aee8d6a4cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:42.988158 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-a65cd3b3-ab16-473e-b1c1-8aee8d6a4cb4 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:42.989563 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a65cd3b3-ab16-473e-b1c1-8aee8d6a4cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:43.012855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36051da-cf20-4a03-b74f-a584e4307a86 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:43.062879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 90/162] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:43.063829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cd3b3-ab16-473e-b1c1-8aee8d6a4cb4 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:43.112208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e58b90-1748-4baf-8e81-752278300eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:43.113253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e58b90-1748-4baf-8e81-752278300eab None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:43.113711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e58b90-1748-4baf-8e81-752278300eab None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:43.114149 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e58b90-1748-4baf-8e81-752278300eab None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:43.115026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e58b90-1748-4baf-8e81-752278300eab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:43.115514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e58b90-1748-4baf-8e81-752278300eab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:43.117091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e58b90-1748-4baf-8e81-752278300eab None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:43.139090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:43.141704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 73/163] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:36:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:43.147322 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e58b90-1748-4baf-8e81-752278300eab None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:43.172447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cd3b3-ab16-473e-b1c1-8aee8d6a4cb4 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:43.213517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c527cc4-9d6c-43a5-8e1d-17ef9dfe4a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:43.214356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c527cc4-9d6c-43a5-8e1d-17ef9dfe4a53 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:43.214792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c527cc4-9d6c-43a5-8e1d-17ef9dfe4a53 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:43.215196 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c527cc4-9d6c-43a5-8e1d-17ef9dfe4a53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:43.216099 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c527cc4-9d6c-43a5-8e1d-17ef9dfe4a53 None admin] RBAC: Adding query filter params (role.id=43667d074eda4143bf3b7f721e83810c, scope.project.id=8b0387feea344c35a5907f20e962c357, user.id=4694cfe2faf74f7085cc07187b0f047f) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:43.216537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c527cc4-9d6c-43a5-8e1d-17ef9dfe4a53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:43.218157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c527cc4-9d6c-43a5-8e1d-17ef9dfe4a53 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:43.224925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 91/164] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:43 2022] GET /identity/v3/role_assignments?user.id=4694cfe2faf74f7085cc07187b0f047f&scope.project.id=8b0387feea344c35a5907f20e962c357&role.id=43667d074eda4143bf3b7f721e83810c => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:43.237655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:43.331237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e58b90-1748-4baf-8e81-752278300eab None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:43.383816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a7422-e3d7-4b82-a5f5-b5fc3406b73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:43.384458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a7422-e3d7-4b82-a5f5-b5fc3406b73e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:43.384739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a7422-e3d7-4b82-a5f5-b5fc3406b73e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:43.385013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a7422-e3d7-4b82-a5f5-b5fc3406b73e None admin] PATH_INFO: `/v3/users/4694cfe2faf74f7085cc07187b0f047f` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:43.391169 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a7422-e3d7-4b82-a5f5-b5fc3406b73e None admin] RBAC: Authorizing `identity:get_user(user_id=4694cfe2faf74f7085cc07187b0f047f)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:43.392635 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a7422-e3d7-4b82-a5f5-b5fc3406b73e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:43.394263 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 74/165] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:43 2022] GET /identity/v3/users/4694cfe2faf74f7085cc07187b0f047f => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:43.400019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c527cc4-9d6c-43a5-8e1d-17ef9dfe4a53 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:43.454758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a7422-e3d7-4b82-a5f5-b5fc3406b73e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:43.546054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c527cc4-9d6c-43a5-8e1d-17ef9dfe4a53 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:43.588776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43fa6f-b81a-41b4-9263-4c692d77e0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:43.589736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43fa6f-b81a-41b4-9263-4c692d77e0db None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:43.590205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43fa6f-b81a-41b4-9263-4c692d77e0db None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:43.590720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43fa6f-b81a-41b4-9263-4c692d77e0db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:43.594673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:43.661071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43fa6f-b81a-41b4-9263-4c692d77e0db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:43.662568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43fa6f-b81a-41b4-9263-4c692d77e0db None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:43.676080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a7422-e3d7-4b82-a5f5-b5fc3406b73e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:43.692158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:43.732636 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a92b3-b10f-4585-acbc-9b900725455a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:43.733784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a92b3-b10f-4585-acbc-9b900725455a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:43.734264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a92b3-b10f-4585-acbc-9b900725455a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:43.734772 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a92b3-b10f-4585-acbc-9b900725455a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:43.747440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a92b3-b10f-4585-acbc-9b900725455a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-b36a92b3-b10f-4585-acbc-9b900725455a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:43.753387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:43.754267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:43.755296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:43.755296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:43.755296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:43.755296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:43.755296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:43.755296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:43.755296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:43.755296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:43.755296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:43.755296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:43.755296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:43.755296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:36:43.762636 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 75/166] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:36:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:43.767619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a92b3-b10f-4585-acbc-9b900725455a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:43.770634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 92/167] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:43.775768 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43fa6f-b81a-41b4-9263-4c692d77e0db None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:43.812600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:43.898458 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43fa6f-b81a-41b4-9263-4c692d77e0db None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:43.942918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a92b3-b10f-4585-acbc-9b900725455a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:43.977738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fed81-8816-4350-84bd-9bddeec5ce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:43.978774 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fed81-8816-4350-84bd-9bddeec5ce57 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:43.979256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fed81-8816-4350-84bd-9bddeec5ce57 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:43.979725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fed81-8816-4350-84bd-9bddeec5ce57 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:43.981974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fed81-8816-4350-84bd-9bddeec5ce57 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:43.983730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fed81-8816-4350-84bd-9bddeec5ce57 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:43.985794 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 93/168] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:43 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:43.988234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fed81-8816-4350-84bd-9bddeec5ce57 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:44.016034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7257a6-c9fd-4f03-b59c-e9270aba7bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:44.017083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7257a6-c9fd-4f03-b59c-e9270aba7bd6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:44.017594 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7257a6-c9fd-4f03-b59c-e9270aba7bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:44.018066 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7257a6-c9fd-4f03-b59c-e9270aba7bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:44.018966 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7257a6-c9fd-4f03-b59c-e9270aba7bd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:44.019471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7257a6-c9fd-4f03-b59c-e9270aba7bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:44.021044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7257a6-c9fd-4f03-b59c-e9270aba7bd6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:44.033709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 76/169] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:44.041269 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7257a6-c9fd-4f03-b59c-e9270aba7bd6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:44.108014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:44.196811 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fed81-8816-4350-84bd-9bddeec5ce57 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:44.200634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:44.245708 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7257a6-c9fd-4f03-b59c-e9270aba7bd6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:44.266227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265ca80-97f3-48ce-a76a-4be96ffa60ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:44.267248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265ca80-97f3-48ce-a76a-4be96ffa60ef None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:44.267739 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265ca80-97f3-48ce-a76a-4be96ffa60ef None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:44.268245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265ca80-97f3-48ce-a76a-4be96ffa60ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:44.287658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265ca80-97f3-48ce-a76a-4be96ffa60ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-c265ca80-97f3-48ce-a76a-4be96ffa60ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:44.293749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:44.294712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:44.295546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:44.295546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:44.295546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:44.295546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:44.295546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:44.295546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:44.295546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:44.295546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:44.295546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:44.295546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:44.295546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:44.295546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:44.303954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 94/170] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:44.304899 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265ca80-97f3-48ce-a76a-4be96ffa60ef None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:44.307934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423d696-a33a-4af4-9f16-ebe7eaf8b3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:44.309210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423d696-a33a-4af4-9f16-ebe7eaf8b3fe None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:44.309811 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423d696-a33a-4af4-9f16-ebe7eaf8b3fe None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:44.310287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423d696-a33a-4af4-9f16-ebe7eaf8b3fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:44.311260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423d696-a33a-4af4-9f16-ebe7eaf8b3fe None admin] RBAC: Adding query filter params (role.id=96271a2245c144d4b43dea66f18074e8, scope.project.id=8b0387feea344c35a5907f20e962c357, user.id=4694cfe2faf74f7085cc07187b0f047f) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:44.311819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423d696-a33a-4af4-9f16-ebe7eaf8b3fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:44.313486 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423d696-a33a-4af4-9f16-ebe7eaf8b3fe None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:44.320957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 77/171] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:44 2022] GET /identity/v3/role_assignments?user.id=4694cfe2faf74f7085cc07187b0f047f&scope.project.id=8b0387feea344c35a5907f20e962c357&role.id=96271a2245c144d4b43dea66f18074e8 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:44.322161 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423d696-a33a-4af4-9f16-ebe7eaf8b3fe None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:44.532735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:44.537405 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423d696-a33a-4af4-9f16-ebe7eaf8b3fe None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:44.543060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265ca80-97f3-48ce-a76a-4be96ffa60ef None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:44.601163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c24bc4-25ae-4e2e-97d5-4418dd3566fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:44.602460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c24bc4-25ae-4e2e-97d5-4418dd3566fb None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:44.602778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c24bc4-25ae-4e2e-97d5-4418dd3566fb None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:44.603050 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c24bc4-25ae-4e2e-97d5-4418dd3566fb None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:44.603844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c24bc4-25ae-4e2e-97d5-4418dd3566fb None admin] RBAC: Adding query filter params (name=member) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:44.604148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c24bc4-25ae-4e2e-97d5-4418dd3566fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:44.604563 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a9b95-d5ad-4be5-b2b3-425dbb08d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:44.605554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c24bc4-25ae-4e2e-97d5-4418dd3566fb None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:44.605855 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a9b95-d5ad-4be5-b2b3-425dbb08d7ea None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:44.606272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a9b95-d5ad-4be5-b2b3-425dbb08d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:44.606713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a9b95-d5ad-4be5-b2b3-425dbb08d7ea None admin] PATH_INFO: `/v3/users/9947df19bb3b44318e674b5168544c18` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:44.608012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a9b95-d5ad-4be5-b2b3-425dbb08d7ea None admin] RBAC: Authorizing `identity:get_user(user_id=9947df19bb3b44318e674b5168544c18)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:44.609632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a9b95-d5ad-4be5-b2b3-425dbb08d7ea None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:44.611442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 95/172] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:44 2022] GET /identity/v3/users/9947df19bb3b44318e674b5168544c18 => generated 247 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:44.622720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a9b95-d5ad-4be5-b2b3-425dbb08d7ea None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:44.629676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 78/173] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:44.638624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c24bc4-25ae-4e2e-97d5-4418dd3566fb None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:44.698854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:44.794328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c24bc4-25ae-4e2e-97d5-4418dd3566fb None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:44.830139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bd07f-d712-4a36-8a70-ee43e807d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:44.830833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bd07f-d712-4a36-8a70-ee43e807d2e2 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:44.831121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bd07f-d712-4a36-8a70-ee43e807d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:44.831558 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bd07f-d712-4a36-8a70-ee43e807d2e2 None admin] PATH_INFO: `/v3/users/4694cfe2faf74f7085cc07187b0f047f` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:44.832720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bd07f-d712-4a36-8a70-ee43e807d2e2 None admin] RBAC: Authorizing `identity:get_user(user_id=4694cfe2faf74f7085cc07187b0f047f)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:44.834293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bd07f-d712-4a36-8a70-ee43e807d2e2 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:44.837069 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 79/174] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:44 2022] GET /identity/v3/users/4694cfe2faf74f7085cc07187b0f047f => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:44.842539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bd07f-d712-4a36-8a70-ee43e807d2e2 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:44.894865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:44.925607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a9b95-d5ad-4be5-b2b3-425dbb08d7ea None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:44.959104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bd07f-d712-4a36-8a70-ee43e807d2e2 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:44.994275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f72dbe-31f6-40af-bf88-71a040c09732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:44.994933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72dbe-31f6-40af-bf88-71a040c09732 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:44.995216 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72dbe-31f6-40af-bf88-71a040c09732 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:44.995601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72dbe-31f6-40af-bf88-71a040c09732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:44.998753 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905cdc7-42ff-4824-b31d-c9831eb9613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:44.999372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905cdc7-42ff-4824-b31d-c9831eb9613e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:44.999650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905cdc7-42ff-4824-b31d-c9831eb9613e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:44.999919 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905cdc7-42ff-4824-b31d-c9831eb9613e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:45.067852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f72dbe-31f6-40af-bf88-71a040c09732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:45.069480 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f72dbe-31f6-40af-bf88-71a040c09732 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:45.073543 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905cdc7-42ff-4824-b31d-c9831eb9613e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:45.075030 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905cdc7-42ff-4824-b31d-c9831eb9613e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:45.103428 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:45.105397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:45.173867 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 96/175] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:45.175240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a231-5324-47bb-8dc9-912b2fe66941 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:45.175525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a231-5324-47bb-8dc9-912b2fe66941 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:45.175796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a231-5324-47bb-8dc9-912b2fe66941 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:45.177207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 97/176] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:45.178034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686053b1-a4ce-4387-ab27-da4197c3bdc4 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:45.178312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686053b1-a4ce-4387-ab27-da4197c3bdc4 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:45.178678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686053b1-a4ce-4387-ab27-da4197c3bdc4 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:45.180023 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 98/177] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:45.181571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686053b1-a4ce-4387-ab27-da4197c3bdc4 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:45.185262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 80/178] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 12:36:45.186375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289c431-c0a9-47e4-910d-cc894cd9d830 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:45.186681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289c431-c0a9-47e4-910d-cc894cd9d830 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:45.186951 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289c431-c0a9-47e4-910d-cc894cd9d830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:45.212688 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-9289c431-c0a9-47e4-910d-cc894cd9d830 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:45.213893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9289c431-c0a9-47e4-910d-cc894cd9d830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:45.300016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 81/179] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:45.307732 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cecff3-b51f-4597-a23e-a507c638fd93 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:45.308022 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cecff3-b51f-4597-a23e-a507c638fd93 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:45.308296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cecff3-b51f-4597-a23e-a507c638fd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:45.316352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686053b1-a4ce-4387-ab27-da4197c3bdc4 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:45.342370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-69cecff3-b51f-4597-a23e-a507c638fd93 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:45.343561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-69cecff3-b51f-4597-a23e-a507c638fd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:45.350137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:45.357338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2d25d-bb43-4cef-b9f4-a9fd77df374e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:45.357994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2d25d-bb43-4cef-b9f4-a9fd77df374e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:45.358277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2d25d-bb43-4cef-b9f4-a9fd77df374e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:45.358582 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2d25d-bb43-4cef-b9f4-a9fd77df374e None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:45.361882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2d25d-bb43-4cef-b9f4-a9fd77df374e None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:45.361882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2d25d-bb43-4cef-b9f4-a9fd77df374e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:45.364278 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 99/180] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:45 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:45.365554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2d25d-bb43-4cef-b9f4-a9fd77df374e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:45.455166 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:45.491644 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 82/181] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:45.492923 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cecff3-b51f-4597-a23e-a507c638fd93 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:45.496432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2d25d-bb43-4cef-b9f4-a9fd77df374e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:45.537105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c65b9-6255-4d0b-94b7-7543a5c1905a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:45.537745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c65b9-6255-4d0b-94b7-7543a5c1905a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:45.538026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c65b9-6255-4d0b-94b7-7543a5c1905a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:45.538300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c65b9-6255-4d0b-94b7-7543a5c1905a None admin] PATH_INFO: `/v3/projects/b68edeee792348298de0934e067273bf` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:45.540019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c65b9-6255-4d0b-94b7-7543a5c1905a None admin] RBAC: Authorizing `identity:get_project(project_id=b68edeee792348298de0934e067273bf)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:45.541454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c65b9-6255-4d0b-94b7-7543a5c1905a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:45.543290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 100/182] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:45 2022] GET /identity/v3/projects/b68edeee792348298de0934e067273bf => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:45.544934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022d52d-fe14-45d5-b4bb-75f77907d2b9 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:45.545364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022d52d-fe14-45d5-b4bb-75f77907d2b9 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:45.545809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022d52d-fe14-45d5-b4bb-75f77907d2b9 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:45.547181 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 101/183] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:45.549187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022d52d-fe14-45d5-b4bb-75f77907d2b9 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:45.631355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:45.679142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022d52d-fe14-45d5-b4bb-75f77907d2b9 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:45.683903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cecff3-b51f-4597-a23e-a507c638fd93 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:45.724152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224e15e-1b01-4617-a075-10966ceae0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:45.772672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224e15e-1b01-4617-a075-10966ceae0a1 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:45.772672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224e15e-1b01-4617-a075-10966ceae0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:45.772672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224e15e-1b01-4617-a075-10966ceae0a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:45.772967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11041258-793c-40fc-97af-00a928d13601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:45.773071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224e15e-1b01-4617-a075-10966ceae0a1 None admin] RBAC: Adding query filter params (role.id=43667d074eda4143bf3b7f721e83810c, scope.project.id=b68edeee792348298de0934e067273bf, user.id=4694cfe2faf74f7085cc07187b0f047f) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:45.773071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224e15e-1b01-4617-a075-10966ceae0a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:45.773205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11041258-793c-40fc-97af-00a928d13601 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:45.773205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11041258-793c-40fc-97af-00a928d13601 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:45.773205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11041258-793c-40fc-97af-00a928d13601 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:45.773432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224e15e-1b01-4617-a075-10966ceae0a1 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:45.773515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11041258-793c-40fc-97af-00a928d13601 None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=8b0387feea344c35a5907f20e962c357, user.id=9947df19bb3b44318e674b5168544c18) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:45.773515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11041258-793c-40fc-97af-00a928d13601 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:45.773515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11041258-793c-40fc-97af-00a928d13601 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:45.773707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 102/184] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:45 2022] GET /identity/v3/role_assignments?user.id=4694cfe2faf74f7085cc07187b0f047f&scope.project.id=b68edeee792348298de0934e067273bf&role.id=43667d074eda4143bf3b7f721e83810c => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:45.773707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85a869-5f35-4efe-b91a-9a7eb9108a7b None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:45.773707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85a869-5f35-4efe-b91a-9a7eb9108a7b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:45.773707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85a869-5f35-4efe-b91a-9a7eb9108a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:45.774012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 83/185] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:45 2022] GET /identity/v3/role_assignments?user.id=9947df19bb3b44318e674b5168544c18&scope.project.id=8b0387feea344c35a5907f20e962c357&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:45.774012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fe91a-f91f-4912-9656-68b942c0286e None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:45.774012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fe91a-f91f-4912-9656-68b942c0286e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:45.774012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fe91a-f91f-4912-9656-68b942c0286e None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:45.774012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 84/186] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:45.774012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fe91a-f91f-4912-9656-68b942c0286e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:45.794358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-ba85a869-5f35-4efe-b91a-9a7eb9108a7b None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:45.796477 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ba85a869-5f35-4efe-b91a-9a7eb9108a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:45.853534 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:45.879021 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 103/187] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:45.880689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682091a-4a37-47bf-b81e-212c5d8e862c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:45.881123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682091a-4a37-47bf-b81e-212c5d8e862c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:45.881666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682091a-4a37-47bf-b81e-212c5d8e862c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:45.903206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d682091a-4a37-47bf-b81e-212c5d8e862c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:45.904755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d682091a-4a37-47bf-b81e-212c5d8e862c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:45.946758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fe91a-f91f-4912-9656-68b942c0286e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:45.990887 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db782275-93cf-494e-801c-d8f0e755cb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:45.993033 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db782275-93cf-494e-801c-d8f0e755cb7e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:45.993811 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db782275-93cf-494e-801c-d8f0e755cb7e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:45.994559 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db782275-93cf-494e-801c-d8f0e755cb7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:46.002873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:46.011996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db782275-93cf-494e-801c-d8f0e755cb7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:46.018702 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 104/188] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:46.018702 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682091a-4a37-47bf-b81e-212c5d8e862c None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-db782275-93cf-494e-801c-d8f0e755cb7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:46.031209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:46.032195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 85/189] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:36:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:46.033015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db782275-93cf-494e-801c-d8f0e755cb7e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:46.159318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682091a-4a37-47bf-b81e-212c5d8e862c None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:46.163018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db782275-93cf-494e-801c-d8f0e755cb7e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:46.178173 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:46.204125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e15f3-9087-4d7e-8454-6f3dfb736685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:46.205152 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e15f3-9087-4d7e-8454-6f3dfb736685 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:46.205637 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e15f3-9087-4d7e-8454-6f3dfb736685 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:46.206109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e15f3-9087-4d7e-8454-6f3dfb736685 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:46.206993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e15f3-9087-4d7e-8454-6f3dfb736685 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:46.207509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e15f3-9087-4d7e-8454-6f3dfb736685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:46.209090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e15f3-9087-4d7e-8454-6f3dfb736685 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:46.213503 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6ea11-fb76-4998-83a6-84c3cab6f1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:46.214148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6ea11-fb76-4998-83a6-84c3cab6f1a8 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:46.214430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6ea11-fb76-4998-83a6-84c3cab6f1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:46.214731 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6ea11-fb76-4998-83a6-84c3cab6f1a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:46.221707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 86/190] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:36:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:46.227164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e15f3-9087-4d7e-8454-6f3dfb736685 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:46.231038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6ea11-fb76-4998-83a6-84c3cab6f1a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-61c6ea11-fb76-4998-83a6-84c3cab6f1a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:46.237073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:46.237992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:46.239268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:46.239268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:46.239268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:46.239268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:46.239268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:46.239268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:46.239268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:46.239268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:46.239268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:46.239268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:46.239268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:46.239268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:46.239268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 105/191] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:36:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:46.243733 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6ea11-fb76-4998-83a6-84c3cab6f1a8 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:46.319411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:46.360733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e15f3-9087-4d7e-8454-6f3dfb736685 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:46.366874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6ea11-fb76-4998-83a6-84c3cab6f1a8 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:46.394609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3cc28-b9d2-457e-8f50-4e4a382cfbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:46.395241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3cc28-b9d2-457e-8f50-4e4a382cfbdb None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:46.395521 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3cc28-b9d2-457e-8f50-4e4a382cfbdb None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:46.395795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3cc28-b9d2-457e-8f50-4e4a382cfbdb None admin] PATH_INFO: `/v3/users/6a69eddf36e84f3e9376277156d7e835` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:46.400445 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7dd53-3293-46ed-8a49-73407af87932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:46.401061 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7dd53-3293-46ed-8a49-73407af87932 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:46.401334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7dd53-3293-46ed-8a49-73407af87932 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:46.401600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7dd53-3293-46ed-8a49-73407af87932 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:46.402717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e7dd53-3293-46ed-8a49-73407af87932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:46.404138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e7dd53-3293-46ed-8a49-73407af87932 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:46.405859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 106/192] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:46.406531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7dd53-3293-46ed-8a49-73407af87932 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:46.408975 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3cc28-b9d2-457e-8f50-4e4a382cfbdb None admin] RBAC: Authorizing `identity:get_user(user_id=6a69eddf36e84f3e9376277156d7e835)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:46.410464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3cc28-b9d2-457e-8f50-4e4a382cfbdb None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:46.417050 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 87/193] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:46 2022] GET /identity/v3/users/6a69eddf36e84f3e9376277156d7e835 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:46.417050 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3cc28-b9d2-457e-8f50-4e4a382cfbdb None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:46.446770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:46.507324 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7dd53-3293-46ed-8a49-73407af87932 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:46.513341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3cc28-b9d2-457e-8f50-4e4a382cfbdb None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:46.526404 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:46.540154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd96386-1d1d-4a0b-a2cb-271af1f6a19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:46.540756 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd96386-1d1d-4a0b-a2cb-271af1f6a19d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:46.541034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd96386-1d1d-4a0b-a2cb-271af1f6a19d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:46.541302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd96386-1d1d-4a0b-a2cb-271af1f6a19d None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:46.541961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd96386-1d1d-4a0b-a2cb-271af1f6a19d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:46.542261 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd96386-1d1d-4a0b-a2cb-271af1f6a19d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:46.543651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd96386-1d1d-4a0b-a2cb-271af1f6a19d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:46.547460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33480b4-80a9-421b-9ea1-0db4079e1814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:46.548078 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33480b4-80a9-421b-9ea1-0db4079e1814 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:46.548355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33480b4-80a9-421b-9ea1-0db4079e1814 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:46.548632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33480b4-80a9-421b-9ea1-0db4079e1814 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:46.551501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 107/194] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:46.552269 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd96386-1d1d-4a0b-a2cb-271af1f6a19d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:46.564533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33480b4-80a9-421b-9ea1-0db4079e1814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-b33480b4-80a9-421b-9ea1-0db4079e1814 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:46.569303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:46.570409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:46.571229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:46.571229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:46.571229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:46.571229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:46.571229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:46.571229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:46.571229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:46.571229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:46.571229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:46.571229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:46.571229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:36:46.571229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:36:46.571229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 88/195] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:46.571921 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33480b4-80a9-421b-9ea1-0db4079e1814 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:46.651793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd96386-1d1d-4a0b-a2cb-271af1f6a19d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:46.663972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:46.673332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33480b4-80a9-421b-9ea1-0db4079e1814 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:46.685646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05f4b0-e8cc-446c-9942-475d5736d0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:46.686292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05f4b0-e8cc-446c-9942-475d5736d0d7 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:46.686610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05f4b0-e8cc-446c-9942-475d5736d0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:46.686883 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05f4b0-e8cc-446c-9942-475d5736d0d7 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:46.687530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05f4b0-e8cc-446c-9942-475d5736d0d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:46.688968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05f4b0-e8cc-446c-9942-475d5736d0d7 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:46.707812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4cfb2-4b79-4ce0-aa83-9aa25e4be3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:46.708423 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4cfb2-4b79-4ce0-aa83-9aa25e4be3a5 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:46.708701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4cfb2-4b79-4ce0-aa83-9aa25e4be3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:46.708996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4cfb2-4b79-4ce0-aa83-9aa25e4be3a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:46.774936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4cfb2-4b79-4ce0-aa83-9aa25e4be3a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:46.776403 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4cfb2-4b79-4ce0-aa83-9aa25e4be3a5 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:46.795039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 108/196] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:36:46 2022] POST /identity/v3/users => generated 297 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:36:46.796066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05f4b0-e8cc-446c-9942-475d5736d0d7 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:46.858726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:46.876555 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:46.892323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 89/197] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:46.892323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4cfb2-4b79-4ce0-aa83-9aa25e4be3a5 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:46.939816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05f4b0-e8cc-446c-9942-475d5736d0d7 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:46.977822 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6affc2-2859-4a05-9f67-e36098fef9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:46.978463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6affc2-2859-4a05-9f67-e36098fef9c2 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:46.978774 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6affc2-2859-4a05-9f67-e36098fef9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:46.979042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6affc2-2859-4a05-9f67-e36098fef9c2 None admin] PATH_INFO: `/v3/users/6a69eddf36e84f3e9376277156d7e835` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:46.980180 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6affc2-2859-4a05-9f67-e36098fef9c2 None admin] RBAC: Authorizing `identity:get_user(user_id=6a69eddf36e84f3e9376277156d7e835)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:46.981602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6affc2-2859-4a05-9f67-e36098fef9c2 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:46.983187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 109/198] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:46 2022] GET /identity/v3/users/6a69eddf36e84f3e9376277156d7e835 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:46.984028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6affc2-2859-4a05-9f67-e36098fef9c2 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:47.004077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4cfb2-4b79-4ce0-aa83-9aa25e4be3a5 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:47.051902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa3261-ca4a-4bba-89b6-e093e1972f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:47.052543 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa3261-ca4a-4bba-89b6-e093e1972f8c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:47.052829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa3261-ca4a-4bba-89b6-e093e1972f8c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:47.053101 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa3261-ca4a-4bba-89b6-e093e1972f8c None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:47.053820 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa3261-ca4a-4bba-89b6-e093e1972f8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:47.054125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa3261-ca4a-4bba-89b6-e093e1972f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:47.056745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa3261-ca4a-4bba-89b6-e093e1972f8c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:47.067274 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 90/199] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:47.068109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa3261-ca4a-4bba-89b6-e093e1972f8c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:47.112040 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6affc2-2859-4a05-9f67-e36098fef9c2 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:47.121262 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:47.142337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:47.157744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7738c20-93cc-44c4-b3c0-03d9cc1f9496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:47.158431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7738c20-93cc-44c4-b3c0-03d9cc1f9496 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:47.159559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7738c20-93cc-44c4-b3c0-03d9cc1f9496 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:47.159559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7738c20-93cc-44c4-b3c0-03d9cc1f9496 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:47.160240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7738c20-93cc-44c4-b3c0-03d9cc1f9496 None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:47.161660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7738c20-93cc-44c4-b3c0-03d9cc1f9496 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:47.163662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 110/200] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:46 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:47.165183 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7738c20-93cc-44c4-b3c0-03d9cc1f9496 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:47.197793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa3261-ca4a-4bba-89b6-e093e1972f8c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:47.242653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48ba5a-bd94-4da0-82c9-c294451a6957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:47.243783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48ba5a-bd94-4da0-82c9-c294451a6957 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:47.244078 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48ba5a-bd94-4da0-82c9-c294451a6957 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:47.244346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48ba5a-bd94-4da0-82c9-c294451a6957 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:47.296868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7738c20-93cc-44c4-b3c0-03d9cc1f9496 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:47.326520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48ba5a-bd94-4da0-82c9-c294451a6957 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:47.328025 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48ba5a-bd94-4da0-82c9-c294451a6957 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:47.346829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6d46a-6d9d-40e7-9b38-cf584f54b7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:47.347504 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6d46a-6d9d-40e7-9b38-cf584f54b7f3 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:47.348123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6d46a-6d9d-40e7-9b38-cf584f54b7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:47.348397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6d46a-6d9d-40e7-9b38-cf584f54b7f3 None admin] PATH_INFO: `/v3/users/158e69e61f0840ed8ad8b22fe178f423` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:47.349536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6d46a-6d9d-40e7-9b38-cf584f54b7f3 None admin] RBAC: Authorizing `identity:get_user(user_id=158e69e61f0840ed8ad8b22fe178f423)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:47.352173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6d46a-6d9d-40e7-9b38-cf584f54b7f3 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:47.354986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 111/201] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:47 2022] GET /identity/v3/users/158e69e61f0840ed8ad8b22fe178f423 => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:47.356282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6d46a-6d9d-40e7-9b38-cf584f54b7f3 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:47.388189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:47.418225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:47.450681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 91/202] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:47.450974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48ba5a-bd94-4da0-82c9-c294451a6957 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:47.486187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6d46a-6d9d-40e7-9b38-cf584f54b7f3 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:47.526449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee11dd-5737-4f25-a295-727ffbedc7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:47.527946 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee11dd-5737-4f25-a295-727ffbedc7ca None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:47.528693 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee11dd-5737-4f25-a295-727ffbedc7ca None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:47.529144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee11dd-5737-4f25-a295-727ffbedc7ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:47.530311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee11dd-5737-4f25-a295-727ffbedc7ca None admin] RBAC: Adding query filter params (role.id=96271a2245c144d4b43dea66f18074e8, scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30, user.id=6a69eddf36e84f3e9376277156d7e835) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:47.531247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee11dd-5737-4f25-a295-727ffbedc7ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:47.533388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee11dd-5737-4f25-a295-727ffbedc7ca None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:47.542590 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 112/203] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:47 2022] GET /identity/v3/role_assignments?user.id=6a69eddf36e84f3e9376277156d7e835&scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30&role.id=96271a2245c144d4b43dea66f18074e8 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:47.543660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee11dd-5737-4f25-a295-727ffbedc7ca None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:47.638034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48ba5a-bd94-4da0-82c9-c294451a6957 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:47.687296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abe5a7-fa79-4548-ace0-0cc8a273f12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:47.692714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abe5a7-fa79-4548-ace0-0cc8a273f12e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:47.692714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abe5a7-fa79-4548-ace0-0cc8a273f12e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:47.692714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abe5a7-fa79-4548-ace0-0cc8a273f12e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:47.724388 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:47.774821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee11dd-5737-4f25-a295-727ffbedc7ca None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:47.784159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:47.784430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abe5a7-fa79-4548-ace0-0cc8a273f12e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:47.785907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abe5a7-fa79-4548-ace0-0cc8a273f12e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:47.839130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa8367-3f7b-48c6-bf18-935a59842d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:47.839764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa8367-3f7b-48c6-bf18-935a59842d04 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:47.840042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa8367-3f7b-48c6-bf18-935a59842d04 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:47.840307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa8367-3f7b-48c6-bf18-935a59842d04 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:47.842518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa8367-3f7b-48c6-bf18-935a59842d04 None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:47.843972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa8367-3f7b-48c6-bf18-935a59842d04 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:47.845771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 113/204] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:47 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:47.851395 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa8367-3f7b-48c6-bf18-935a59842d04 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:48.006260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 92/205] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:48.012837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abe5a7-fa79-4548-ace0-0cc8a273f12e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:48.027559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa8367-3f7b-48c6-bf18-935a59842d04 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:48.086819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6a0e6-8d16-4184-b1e9-039faa7af191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:48.087780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6a0e6-8d16-4184-b1e9-039faa7af191 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:48.088271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6a0e6-8d16-4184-b1e9-039faa7af191 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:48.088751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6a0e6-8d16-4184-b1e9-039faa7af191 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:48.089671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6a0e6-8d16-4184-b1e9-039faa7af191 None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30, user.id=6a69eddf36e84f3e9376277156d7e835) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:48.090170 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6a0e6-8d16-4184-b1e9-039faa7af191 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:48.091830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6a0e6-8d16-4184-b1e9-039faa7af191 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:48.104287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 114/206] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:47 2022] GET /identity/v3/role_assignments?user.id=6a69eddf36e84f3e9376277156d7e835&scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:48.123212 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:48.165348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abe5a7-fa79-4548-ace0-0cc8a273f12e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:48.217772 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d72c2-7a88-4721-add8-5ca105b59f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:48.218432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d72c2-7a88-4721-add8-5ca105b59f50 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:48.218741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d72c2-7a88-4721-add8-5ca105b59f50 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:48.219012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d72c2-7a88-4721-add8-5ca105b59f50 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:48.223238 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d72c2-7a88-4721-add8-5ca105b59f50 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:48.224739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d72c2-7a88-4721-add8-5ca105b59f50 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:48.227480 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 93/207] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:48 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:48.233601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6a0e6-8d16-4184-b1e9-039faa7af191 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:48.305698 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:48.372938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6a0e6-8d16-4184-b1e9-039faa7af191 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:48.413094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29426f-07a1-4f50-ba98-2a9e6bb61107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:48.413770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29426f-07a1-4f50-ba98-2a9e6bb61107 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:48.414124 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29426f-07a1-4f50-ba98-2a9e6bb61107 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:48.414396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29426f-07a1-4f50-ba98-2a9e6bb61107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:48.415194 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29426f-07a1-4f50-ba98-2a9e6bb61107 None admin] RBAC: Adding query filter params (role.id=ca39d64092424a34aaaf4818ca16e988, scope.project.id=8b0387feea344c35a5907f20e962c357, user.id=158e69e61f0840ed8ad8b22fe178f423) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:48.415547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29426f-07a1-4f50-ba98-2a9e6bb61107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:48.416974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29426f-07a1-4f50-ba98-2a9e6bb61107 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:48.425089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 115/208] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:48 2022] GET /identity/v3/role_assignments?user.id=158e69e61f0840ed8ad8b22fe178f423&scope.project.id=8b0387feea344c35a5907f20e962c357&role.id=ca39d64092424a34aaaf4818ca16e988 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:49.309913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee598e32-f8c9-4cea-9a8a-09f197b83c95 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:49.310960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee598e32-f8c9-4cea-9a8a-09f197b83c95 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:49.311437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee598e32-f8c9-4cea-9a8a-09f197b83c95 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:49.313222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 94/209] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:49.318899 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7ae49-8109-4352-b5fb-0f12b70cd64f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:49.319704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7ae49-8109-4352-b5fb-0f12b70cd64f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:49.320203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7ae49-8109-4352-b5fb-0f12b70cd64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:49.340013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-4be7ae49-8109-4352-b5fb-0f12b70cd64f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:49.341505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4be7ae49-8109-4352-b5fb-0f12b70cd64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:49.415026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 116/210] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:36:49.573833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad45d8-c792-43fa-b481-2b7163baaf05 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:49.574150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad45d8-c792-43fa-b481-2b7163baaf05 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:49.574391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad45d8-c792-43fa-b481-2b7163baaf05 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:49.576052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 95/211] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:49.581299 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e89208-ae83-46dc-b243-f24310b59a43 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:49.581591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e89208-ae83-46dc-b243-f24310b59a43 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:49.581867 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e89208-ae83-46dc-b243-f24310b59a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:49.600117 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d3e89208-ae83-46dc-b243-f24310b59a43 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:49.601293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d3e89208-ae83-46dc-b243-f24310b59a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:49.628124 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:49.669772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 117/212] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:49.692488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad45d8-c792-43fa-b481-2b7163baaf05 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:49.811964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad45d8-c792-43fa-b481-2b7163baaf05 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:49.845816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:49.859674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b4d17-c1a2-4ab1-9afb-f3718962fee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:49.860344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b4d17-c1a2-4ab1-9afb-f3718962fee5 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:49.860632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b4d17-c1a2-4ab1-9afb-f3718962fee5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:49.860907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b4d17-c1a2-4ab1-9afb-f3718962fee5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:49.877833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b4d17-c1a2-4ab1-9afb-f3718962fee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-f20b4d17-c1a2-4ab1-9afb-f3718962fee5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:49.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:49.884377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:49.885208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:49.885208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:49.885208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:49.885208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:49.885208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:49.885208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:49.885208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:49.885208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:49.885208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:49.885208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:49.885208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:49.885208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:36:49.887325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 96/213] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:49.893304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e89208-ae83-46dc-b243-f24310b59a43 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:49.962043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b4d17-c1a2-4ab1-9afb-f3718962fee5 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:50.003742 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e89208-ae83-46dc-b243-f24310b59a43 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:50.045441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef2c17-8407-4056-9c5e-cf3e4b2564a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:50.046446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef2c17-8407-4056-9c5e-cf3e4b2564a3 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:50.046956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef2c17-8407-4056-9c5e-cf3e4b2564a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:50.047427 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef2c17-8407-4056-9c5e-cf3e4b2564a3 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:50.048302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef2c17-8407-4056-9c5e-cf3e4b2564a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:50.048820 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef2c17-8407-4056-9c5e-cf3e4b2564a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:50.050368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef2c17-8407-4056-9c5e-cf3e4b2564a3 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:50.060234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 118/214] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:50.060234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdb934-a274-4f83-8ac8-e71f2d9657c5 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:50.060234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdb934-a274-4f83-8ac8-e71f2d9657c5 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:50.060234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdb934-a274-4f83-8ac8-e71f2d9657c5 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:50.060234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 119/215] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:50.068106 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdb934-a274-4f83-8ac8-e71f2d9657c5 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:50.079345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b4d17-c1a2-4ab1-9afb-f3718962fee5 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:50.095385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:50.125202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0379f3f-904a-493e-b0a0-959e9e5f2ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:50.125869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0379f3f-904a-493e-b0a0-959e9e5f2ef8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:50.126149 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0379f3f-904a-493e-b0a0-959e9e5f2ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:50.126418 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0379f3f-904a-493e-b0a0-959e9e5f2ef8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:50.127561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0379f3f-904a-493e-b0a0-959e9e5f2ef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:50.129040 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0379f3f-904a-493e-b0a0-959e9e5f2ef8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:50.130858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 97/216] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:50.136705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b0714-5a39-4902-b85d-fba7af4e9216 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:50.136994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b0714-5a39-4902-b85d-fba7af4e9216 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:50.137266 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b0714-5a39-4902-b85d-fba7af4e9216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:50.154856 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-475b0714-5a39-4902-b85d-fba7af4e9216 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:50.156024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-475b0714-5a39-4902-b85d-fba7af4e9216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:50.182422 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdb934-a274-4f83-8ac8-e71f2d9657c5 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:50.203084 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:50.218266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cf55f-b673-4222-8547-69891280232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:50.218951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cf55f-b673-4222-8547-69891280232e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:50.219228 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cf55f-b673-4222-8547-69891280232e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:50.219504 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cf55f-b673-4222-8547-69891280232e None admin] PATH_INFO: `/v3/users/40ffc95ccbce4d12b0db524b56bc5436` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:50.224362 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 98/217] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:50.234697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b0714-5a39-4902-b85d-fba7af4e9216 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:50.235296 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cf55f-b673-4222-8547-69891280232e None admin] RBAC: Authorizing `identity:get_user(user_id=40ffc95ccbce4d12b0db524b56bc5436)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:50.236747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cf55f-b673-4222-8547-69891280232e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:50.238592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 120/218] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:50 2022] GET /identity/v3/users/40ffc95ccbce4d12b0db524b56bc5436 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:50.246804 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cf55f-b673-4222-8547-69891280232e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:50.346718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:50.377617 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cf55f-b673-4222-8547-69891280232e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:50.379210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b0714-5a39-4902-b85d-fba7af4e9216 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:50.419805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e59657-26e2-4c56-9f0e-a22023b257af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:50.420623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e59657-26e2-4c56-9f0e-a22023b257af None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:50.421048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e59657-26e2-4c56-9f0e-a22023b257af None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:50.421516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e59657-26e2-4c56-9f0e-a22023b257af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:50.427451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b29dc-e8c5-4245-aeaf-3b4a71fd3148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:50.428267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b29dc-e8c5-4245-aeaf-3b4a71fd3148 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:50.428691 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b29dc-e8c5-4245-aeaf-3b4a71fd3148 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:50.429155 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b29dc-e8c5-4245-aeaf-3b4a71fd3148 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:50.429978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b29dc-e8c5-4245-aeaf-3b4a71fd3148 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:50.432180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b29dc-e8c5-4245-aeaf-3b4a71fd3148 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:50.491326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e59657-26e2-4c56-9f0e-a22023b257af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:50.493309 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e59657-26e2-4c56-9f0e-a22023b257af None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:50.538004 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 99/219] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:36:50 2022] POST /identity/v3/users => generated 297 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:36:50.539897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3662960-52a4-449c-a6fc-eb42d069f19e None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:50.540181 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3662960-52a4-449c-a6fc-eb42d069f19e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:50.540453 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3662960-52a4-449c-a6fc-eb42d069f19e None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:50.541678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 100/220] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:50.542454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3662960-52a4-449c-a6fc-eb42d069f19e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:50.584645 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:50.591476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:50.613173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 121/221] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:50.614757 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a32a91-f653-41ef-8b1f-9afee9ad984e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:50.615095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a32a91-f653-41ef-8b1f-9afee9ad984e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:50.615373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a32a91-f653-41ef-8b1f-9afee9ad984e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:50.635571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-f9a32a91-f653-41ef-8b1f-9afee9ad984e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:50.637011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f9a32a91-f653-41ef-8b1f-9afee9ad984e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:50.667164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3662960-52a4-449c-a6fc-eb42d069f19e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:50.706142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27379f1-9c3b-4427-ba4b-a980c5afc51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:50.707240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27379f1-9c3b-4427-ba4b-a980c5afc51b None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:50.707696 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27379f1-9c3b-4427-ba4b-a980c5afc51b None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:50.708175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27379f1-9c3b-4427-ba4b-a980c5afc51b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:50.713404 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 122/222] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:50.714606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a32a91-f653-41ef-8b1f-9afee9ad984e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:50.730360 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27379f1-9c3b-4427-ba4b-a980c5afc51b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-b27379f1-9c3b-4427-ba4b-a980c5afc51b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:50.736618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:50.737548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:50.738341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:50.738341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:50.738341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:50.738341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:50.738341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:50.738341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:50.738341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:50.738341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:50.738341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:50.738341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:50.738341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:36:50.738341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:36:50.738341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 101/223] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:36:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:50.745313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27379f1-9c3b-4427-ba4b-a980c5afc51b None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:50.856501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a32a91-f653-41ef-8b1f-9afee9ad984e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:50.873664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:50.877213 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:50.883805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27379f1-9c3b-4427-ba4b-a980c5afc51b None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:50.896601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75789d-176d-40be-8b28-1fdafbfd0a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:50.897236 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75789d-176d-40be-8b28-1fdafbfd0a1d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:50.897505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75789d-176d-40be-8b28-1fdafbfd0a1d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:50.898292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75789d-176d-40be-8b28-1fdafbfd0a1d None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:50.899269 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75789d-176d-40be-8b28-1fdafbfd0a1d None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:50.900474 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75789d-176d-40be-8b28-1fdafbfd0a1d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:50.902262 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 123/224] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:50 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:50.908229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75789d-176d-40be-8b28-1fdafbfd0a1d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:50.928039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7daf106-af49-4e49-be72-e8eb1314f103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:50.928728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7daf106-af49-4e49-be72-e8eb1314f103 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:50.928997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7daf106-af49-4e49-be72-e8eb1314f103 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:50.929271 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7daf106-af49-4e49-be72-e8eb1314f103 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:50.930001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7daf106-af49-4e49-be72-e8eb1314f103 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:50.930362 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7daf106-af49-4e49-be72-e8eb1314f103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:50.931755 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7daf106-af49-4e49-be72-e8eb1314f103 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:50.942553 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 102/225] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:36:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:50.951348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7daf106-af49-4e49-be72-e8eb1314f103 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:51.044090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75789d-176d-40be-8b28-1fdafbfd0a1d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:51.084592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db613ed-27c3-429e-b0e8-cc5b105123ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:51.084825 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7daf106-af49-4e49-be72-e8eb1314f103 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:51.085278 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db613ed-27c3-429e-b0e8-cc5b105123ac None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:51.085558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db613ed-27c3-429e-b0e8-cc5b105123ac None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:51.085825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db613ed-27c3-429e-b0e8-cc5b105123ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:51.088214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db613ed-27c3-429e-b0e8-cc5b105123ac None admin] RBAC: Adding query filter params (role.id=43667d074eda4143bf3b7f721e83810c, scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30, user.id=40ffc95ccbce4d12b0db524b56bc5436) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:51.088525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db613ed-27c3-429e-b0e8-cc5b105123ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:51.089965 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db613ed-27c3-429e-b0e8-cc5b105123ac None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:51.096019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 124/226] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:50 2022] GET /identity/v3/role_assignments?user.id=40ffc95ccbce4d12b0db524b56bc5436&scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30&role.id=43667d074eda4143bf3b7f721e83810c => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:51.096851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db613ed-27c3-429e-b0e8-cc5b105123ac None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:51.137339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916bdd8-20a5-44dd-b125-009e47d4cdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:51.137992 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916bdd8-20a5-44dd-b125-009e47d4cdbe None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:51.138264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916bdd8-20a5-44dd-b125-009e47d4cdbe None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:51.138561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916bdd8-20a5-44dd-b125-009e47d4cdbe None admin] PATH_INFO: `/v3/users/6a69eddf36e84f3e9376277156d7e835` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:51.139757 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916bdd8-20a5-44dd-b125-009e47d4cdbe None admin] RBAC: Authorizing `identity:get_user(user_id=6a69eddf36e84f3e9376277156d7e835)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:51.141497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916bdd8-20a5-44dd-b125-009e47d4cdbe None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:51.148276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 103/227] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:50 2022] GET /identity/v3/users/6a69eddf36e84f3e9376277156d7e835 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:51.151267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916bdd8-20a5-44dd-b125-009e47d4cdbe None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:51.184812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:51.218893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:51.273005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916bdd8-20a5-44dd-b125-009e47d4cdbe None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:51.336604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04865a6-6cdb-446a-a917-a319c175f554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:51.337395 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db613ed-27c3-429e-b0e8-cc5b105123ac None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:51.337939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04865a6-6cdb-446a-a917-a319c175f554 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:51.338412 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04865a6-6cdb-446a-a917-a319c175f554 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:51.338920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04865a6-6cdb-446a-a917-a319c175f554 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:51.340394 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04865a6-6cdb-446a-a917-a319c175f554 None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:51.342340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04865a6-6cdb-446a-a917-a319c175f554 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:51.355011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 104/228] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:51 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:51.370103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04865a6-6cdb-446a-a917-a319c175f554 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:51.382567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e1c01-5bae-45ba-a1ee-666cd34aa39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:51.383850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e1c01-5bae-45ba-a1ee-666cd34aa39e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:51.384289 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e1c01-5bae-45ba-a1ee-666cd34aa39e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:51.384604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e1c01-5bae-45ba-a1ee-666cd34aa39e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:51.416114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e1c01-5bae-45ba-a1ee-666cd34aa39e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-3d6e1c01-5bae-45ba-a1ee-666cd34aa39e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:51.421523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:51.422411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:51.423238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:51.423238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:51.423238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:51.423238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:51.423238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:51.423238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:51.423238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:51.423238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:51.423238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:51.423238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:51.423238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:51.423238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:51.430775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 125/229] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:36:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:51.437105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e1c01-5bae-45ba-a1ee-666cd34aa39e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:51.481179 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:51.488632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04865a6-6cdb-446a-a917-a319c175f554 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:51.526913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204289c-ddff-4bfb-9862-5430e5c9a4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:51.527575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204289c-ddff-4bfb-9862-5430e5c9a4d6 None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:51.527850 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204289c-ddff-4bfb-9862-5430e5c9a4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:51.528118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204289c-ddff-4bfb-9862-5430e5c9a4d6 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30/users/6a69eddf36e84f3e9376277156d7e835/roles/96271a2245c144d4b43dea66f18074e8` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:51.530287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204289c-ddff-4bfb-9862-5430e5c9a4d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=3cab176a56624fcbb4fd9cf5ad72ab30, user_id=6a69eddf36e84f3e9376277156d7e835, role_id=96271a2245c144d4b43dea66f18074e8)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:51.532132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204289c-ddff-4bfb-9862-5430e5c9a4d6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:51.610100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e1c01-5bae-45ba-a1ee-666cd34aa39e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:51.612510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:51.612940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 105/230] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:36:51 2022] PUT /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30/users/6a69eddf36e84f3e9376277156d7e835/roles/96271a2245c144d4b43dea66f18074e8 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:36:51.651212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a2f1b-7546-446e-8076-15e1d824a49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:51.651850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a2f1b-7546-446e-8076-15e1d824a49d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:51.652131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a2f1b-7546-446e-8076-15e1d824a49d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:51.652409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a2f1b-7546-446e-8076-15e1d824a49d None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:51.653278 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a2f1b-7546-446e-8076-15e1d824a49d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:51.653589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a2f1b-7546-446e-8076-15e1d824a49d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:51.655283 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a2f1b-7546-446e-8076-15e1d824a49d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:51.665897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 126/231] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:51.671950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204289c-ddff-4bfb-9862-5430e5c9a4d6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:51.833704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204289c-ddff-4bfb-9862-5430e5c9a4d6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:51.872049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc05878-0294-462e-b7de-bfeef336bb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:51.872611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc05878-0294-462e-b7de-bfeef336bb83 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:51.872889 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc05878-0294-462e-b7de-bfeef336bb83 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:51.873166 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc05878-0294-462e-b7de-bfeef336bb83 None admin] PATH_INFO: `/v3/users/6a69eddf36e84f3e9376277156d7e835` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:51.874290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc05878-0294-462e-b7de-bfeef336bb83 None admin] RBAC: Authorizing `identity:get_user(user_id=6a69eddf36e84f3e9376277156d7e835)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:51.875812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc05878-0294-462e-b7de-bfeef336bb83 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:51.877674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 106/232] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:51 2022] GET /identity/v3/users/6a69eddf36e84f3e9376277156d7e835 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:51.883951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a2f1b-7546-446e-8076-15e1d824a49d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:52.000138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a2f1b-7546-446e-8076-15e1d824a49d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:52.042472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abdce1-4c38-4ae9-97d8-0e69f96d9e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:52.043186 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abdce1-4c38-4ae9-97d8-0e69f96d9e5e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:52.043476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abdce1-4c38-4ae9-97d8-0e69f96d9e5e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:52.043748 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abdce1-4c38-4ae9-97d8-0e69f96d9e5e None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:52.044900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abdce1-4c38-4ae9-97d8-0e69f96d9e5e None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:52.046336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abdce1-4c38-4ae9-97d8-0e69f96d9e5e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:52.048067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 127/233] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:51 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:52.053697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc05878-0294-462e-b7de-bfeef336bb83 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:52.085116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:52.194136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc05878-0294-462e-b7de-bfeef336bb83 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:52.243464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67712f-fc58-4dd9-9ab4-0a03638c340c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:52.244140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67712f-fc58-4dd9-9ab4-0a03638c340c None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:52.244418 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67712f-fc58-4dd9-9ab4-0a03638c340c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:52.244689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67712f-fc58-4dd9-9ab4-0a03638c340c None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30/users/6a69eddf36e84f3e9376277156d7e835/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:52.249562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67712f-fc58-4dd9-9ab4-0a03638c340c None admin] RBAC: Authorizing `identity:create_grant(project_id=3cab176a56624fcbb4fd9cf5ad72ab30, user_id=6a69eddf36e84f3e9376277156d7e835, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:52.251044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67712f-fc58-4dd9-9ab4-0a03638c340c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:52.355756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 107/234] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:36:52 2022] PUT /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30/users/6a69eddf36e84f3e9376277156d7e835/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:36:53.158518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086aa63-6dd5-4f8d-9187-d5160d59940f None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:53.159010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086aa63-6dd5-4f8d-9187-d5160d59940f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:53.159084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086aa63-6dd5-4f8d-9187-d5160d59940f None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:53.160661 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 128/235] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:53.176159 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1523a2-846d-412d-a019-65b756a2e1e1 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:53.176648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1523a2-846d-412d-a019-65b756a2e1e1 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:53.176721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1523a2-846d-412d-a019-65b756a2e1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:53.202103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ac1523a2-846d-412d-a019-65b756a2e1e1 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:53.203318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ac1523a2-846d-412d-a019-65b756a2e1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:53.294284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 108/236] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:53.454039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf3410-47bc-4963-ade0-a97418a928bb None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:53.454621 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf3410-47bc-4963-ade0-a97418a928bb None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:53.455132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf3410-47bc-4963-ade0-a97418a928bb None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:53.456933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 129/237] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:53.464484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e1654-aee3-47c6-a44c-a514ac4e1984 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:53.465027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e1654-aee3-47c6-a44c-a514ac4e1984 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:53.465579 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e1654-aee3-47c6-a44c-a514ac4e1984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:53.488922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-7d2e1654-aee3-47c6-a44c-a514ac4e1984 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:53.490415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7d2e1654-aee3-47c6-a44c-a514ac4e1984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:53.503108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:53.576078 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 109/238] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:53.718675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf3410-47bc-4963-ade0-a97418a928bb None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:53.757186 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:53.830993 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf3410-47bc-4963-ade0-a97418a928bb None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:53.873369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc21bf-ce16-4a99-aaa5-d3d108f3998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:53.874623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc21bf-ce16-4a99-aaa5-d3d108f3998e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:53.875100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc21bf-ce16-4a99-aaa5-d3d108f3998e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:53.875615 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc21bf-ce16-4a99-aaa5-d3d108f3998e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:53.887078 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc21bf-ce16-4a99-aaa5-d3d108f3998e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-1ecc21bf-ce16-4a99-aaa5-d3d108f3998e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:53.892646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:53.893510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:53.894337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:53.894337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:53.894337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:53.894337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:53.894337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:53.894337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:53.894337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:53.894337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:53.894337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:53.894337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:53.894337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:36:53.894337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:53.896655 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 130/239] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:53.903160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e1654-aee3-47c6-a44c-a514ac4e1984 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:54.027664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e1654-aee3-47c6-a44c-a514ac4e1984 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:54.084076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb56ee8-b048-4882-a756-9b35d36fcd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:54.085062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb56ee8-b048-4882-a756-9b35d36fcd2e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:54.085585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb56ee8-b048-4882-a756-9b35d36fcd2e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:54.086722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb56ee8-b048-4882-a756-9b35d36fcd2e None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:54.087636 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb56ee8-b048-4882-a756-9b35d36fcd2e None admin] RBAC: Adding query filter params (name=reader) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:54.088146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb56ee8-b048-4882-a756-9b35d36fcd2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:54.089727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb56ee8-b048-4882-a756-9b35d36fcd2e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:54.091655 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc21bf-ce16-4a99-aaa5-d3d108f3998e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:54.116945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 110/240] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:54.131724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb56ee8-b048-4882-a756-9b35d36fcd2e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:54.136735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:54.252275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:54.261425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc21bf-ce16-4a99-aaa5-d3d108f3998e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:54.271487 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb56ee8-b048-4882-a756-9b35d36fcd2e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:54.307291 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5ebe0-2b34-45e5-af22-5fb25d330316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:54.307777 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5ebe0-2b34-45e5-af22-5fb25d330316 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:54.308059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5ebe0-2b34-45e5-af22-5fb25d330316 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:54.308330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5ebe0-2b34-45e5-af22-5fb25d330316 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:54.309887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5ebe0-2b34-45e5-af22-5fb25d330316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:54.311431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5ebe0-2b34-45e5-af22-5fb25d330316 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:54.313115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 131/241] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:54.318828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026759e-bafc-42e4-9fb8-8be9940a147a None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:54.319113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026759e-bafc-42e4-9fb8-8be9940a147a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:54.319342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f04489-8ca3-44e2-9e34-ba38c1da1252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:54.319466 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026759e-bafc-42e4-9fb8-8be9940a147a None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:54.320308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f04489-8ca3-44e2-9e34-ba38c1da1252 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:54.320818 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 132/242] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:54.321310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f04489-8ca3-44e2-9e34-ba38c1da1252 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:54.321730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f04489-8ca3-44e2-9e34-ba38c1da1252 None admin] PATH_INFO: `/v3/users/991b210436144ad5b367071555b51902` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:54.325661 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026759e-bafc-42e4-9fb8-8be9940a147a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:54.339756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f04489-8ca3-44e2-9e34-ba38c1da1252 None admin] RBAC: Authorizing `identity:get_user(user_id=991b210436144ad5b367071555b51902)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:54.342077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f04489-8ca3-44e2-9e34-ba38c1da1252 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:54.344699 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 111/243] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:54 2022] GET /identity/v3/users/991b210436144ad5b367071555b51902 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:54.346038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8307a6-af07-4a35-982f-19acc4f73981 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:54.346734 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8307a6-af07-4a35-982f-19acc4f73981 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:54.347015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8307a6-af07-4a35-982f-19acc4f73981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:54.367790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-9d8307a6-af07-4a35-982f-19acc4f73981 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:54.368977 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9d8307a6-af07-4a35-982f-19acc4f73981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:54.433217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:54.441613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026759e-bafc-42e4-9fb8-8be9940a147a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:54.443841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 112/244] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:54.444905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e4fd2-60d6-47e3-aa5f-6b365b359df3 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:54.445196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e4fd2-60d6-47e3-aa5f-6b365b359df3 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:54.446030 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e4fd2-60d6-47e3-aa5f-6b365b359df3 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:54.447633 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 113/245] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:54.452235 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e4fd2-60d6-47e3-aa5f-6b365b359df3 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:54.497406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95009e4e-e395-4151-8566-3736517ca3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:54.498017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95009e4e-e395-4151-8566-3736517ca3c6 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:54.498294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95009e4e-e395-4151-8566-3736517ca3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:54.498596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95009e4e-e395-4151-8566-3736517ca3c6 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:54.499351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95009e4e-e395-4151-8566-3736517ca3c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:54.500875 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95009e4e-e395-4151-8566-3736517ca3c6 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:54.577876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e4fd2-60d6-47e3-aa5f-6b365b359df3 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:54.592816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 133/246] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:36:54 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:36:54.594249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef6b68-27e6-4f40-ae2e-8616d6749bd1 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:54.594993 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef6b68-27e6-4f40-ae2e-8616d6749bd1 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:54.595331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef6b68-27e6-4f40-ae2e-8616d6749bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:54.619569 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b2ef6b68-27e6-4f40-ae2e-8616d6749bd1 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:54.621099 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b2ef6b68-27e6-4f40-ae2e-8616d6749bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:54.639469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:54.643297 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a42c6f-896f-43f0-bb51-af01e2e30039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:54.643932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a42c6f-896f-43f0-bb51-af01e2e30039 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:54.644219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a42c6f-896f-43f0-bb51-af01e2e30039 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:54.644492 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a42c6f-896f-43f0-bb51-af01e2e30039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:54.736789 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 134/247] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:54.737778 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef6b68-27e6-4f40-ae2e-8616d6749bd1 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:54.755791 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:54.774517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a42c6f-896f-43f0-bb51-af01e2e30039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:54.774517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a42c6f-896f-43f0-bb51-af01e2e30039 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:54.894124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 114/248] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:54.900438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a42c6f-896f-43f0-bb51-af01e2e30039 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:54.923862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef6b68-27e6-4f40-ae2e-8616d6749bd1 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:54.966467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee84917-9bbb-4680-a7ae-6273a1f9d5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:54.967186 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee84917-9bbb-4680-a7ae-6273a1f9d5da None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:54.967464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee84917-9bbb-4680-a7ae-6273a1f9d5da None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:54.967736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee84917-9bbb-4680-a7ae-6273a1f9d5da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:54.981937 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee84917-9bbb-4680-a7ae-6273a1f9d5da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-9ee84917-9bbb-4680-a7ae-6273a1f9d5da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:54.987425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:54.988420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:54.989232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:54.989232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:54.989232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:54.989232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:54.989232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:54.989232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:54.989232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:54.989232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:54.989232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:54.989232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:54.989232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:36:54.989232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:54.989232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 135/249] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:36:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:54.997697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee84917-9bbb-4680-a7ae-6273a1f9d5da None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:55.013131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:55.022614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:55.035744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a42c6f-896f-43f0-bb51-af01e2e30039 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:55.087188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa6024-4acd-4a79-9fe0-34ccc7712376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:55.088296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa6024-4acd-4a79-9fe0-34ccc7712376 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:55.088782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa6024-4acd-4a79-9fe0-34ccc7712376 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:55.089248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa6024-4acd-4a79-9fe0-34ccc7712376 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:55.090728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa6024-4acd-4a79-9fe0-34ccc7712376 None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:55.092587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa6024-4acd-4a79-9fe0-34ccc7712376 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:55.094916 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 115/250] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:54 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:55.102517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa6024-4acd-4a79-9fe0-34ccc7712376 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:55.138557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee84917-9bbb-4680-a7ae-6273a1f9d5da None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:55.190164 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7dc68-05bc-4b9e-bfc6-5cd6586a1f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:55.190906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7dc68-05bc-4b9e-bfc6-5cd6586a1f69 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:55.191247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7dc68-05bc-4b9e-bfc6-5cd6586a1f69 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:55.191644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7dc68-05bc-4b9e-bfc6-5cd6586a1f69 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:55.192379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab7dc68-05bc-4b9e-bfc6-5cd6586a1f69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:55.192723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab7dc68-05bc-4b9e-bfc6-5cd6586a1f69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:55.194113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab7dc68-05bc-4b9e-bfc6-5cd6586a1f69 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:55.204271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 136/251] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:36:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:55.205029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7dc68-05bc-4b9e-bfc6-5cd6586a1f69 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:55.242916 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa6024-4acd-4a79-9fe0-34ccc7712376 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:55.283433 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d94f81-4802-4497-9770-f537570bbd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:55.284058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d94f81-4802-4497-9770-f537570bbd21 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:55.284332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d94f81-4802-4497-9770-f537570bbd21 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:55.284605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d94f81-4802-4497-9770-f537570bbd21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:55.285769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d94f81-4802-4497-9770-f537570bbd21 None admin] RBAC: Adding query filter params (role.id=ca39d64092424a34aaaf4818ca16e988, scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30, user.id=991b210436144ad5b367071555b51902) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:55.286208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d94f81-4802-4497-9770-f537570bbd21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:55.287682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d94f81-4802-4497-9770-f537570bbd21 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:55.296989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 116/252] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:55 2022] GET /identity/v3/role_assignments?user.id=991b210436144ad5b367071555b51902&scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30&role.id=ca39d64092424a34aaaf4818ca16e988 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:55.297151 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:55.297575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d3e73-95d6-48ff-999c-a1cb8e4302ef None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:55.297575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d3e73-95d6-48ff-999c-a1cb8e4302ef None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:55.297575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d3e73-95d6-48ff-999c-a1cb8e4302ef None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:55.298956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 117/253] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:55.303918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d3e73-95d6-48ff-999c-a1cb8e4302ef None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:55.333904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:55.338650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7dc68-05bc-4b9e-bfc6-5cd6586a1f69 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:55.403146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87afb12-f06b-4a13-9784-e115f8c3e947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:55.405132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87afb12-f06b-4a13-9784-e115f8c3e947 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:55.405415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87afb12-f06b-4a13-9784-e115f8c3e947 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:55.405688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87afb12-f06b-4a13-9784-e115f8c3e947 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:55.433226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87afb12-f06b-4a13-9784-e115f8c3e947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-d87afb12-f06b-4a13-9784-e115f8c3e947 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:55.449442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:55.450799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:55.451618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:55.451618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:55.451618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:55.451618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:55.451618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:55.451618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:55.451618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:55.451618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:55.451618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:55.451618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:55.451618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:55.451618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:55.451618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 137/254] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:55.452287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a38932-a5ae-4bc6-84b0-de920d05ffee None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:55.452883 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a38932-a5ae-4bc6-84b0-de920d05ffee None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:55.452883 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a38932-a5ae-4bc6-84b0-de920d05ffee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:55.495567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-e3a38932-a5ae-4bc6-84b0-de920d05ffee None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:55.496780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e3a38932-a5ae-4bc6-84b0-de920d05ffee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:55.514065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d3e73-95d6-48ff-999c-a1cb8e4302ef None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:55.564293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f5007-7882-4810-89e1-7b65754aa8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:55.564931 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f5007-7882-4810-89e1-7b65754aa8de None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:55.565214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f5007-7882-4810-89e1-7b65754aa8de None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:55.565493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f5007-7882-4810-89e1-7b65754aa8de None admin] PATH_INFO: `/v3/users/6a69eddf36e84f3e9376277156d7e835` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:55.566660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f5007-7882-4810-89e1-7b65754aa8de None admin] RBAC: Authorizing `identity:get_user(user_id=6a69eddf36e84f3e9376277156d7e835)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:55.568103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f5007-7882-4810-89e1-7b65754aa8de None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:55.569701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 118/255] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:55 2022] GET /identity/v3/users/6a69eddf36e84f3e9376277156d7e835 => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:55.571623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f5007-7882-4810-89e1-7b65754aa8de None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:55.611131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 138/256] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:55.612235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a38932-a5ae-4bc6-84b0-de920d05ffee None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:55.704748 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:55.782095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a38932-a5ae-4bc6-84b0-de920d05ffee None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:55.807733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:55.826141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f5007-7882-4810-89e1-7b65754aa8de None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:55.843677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d799d85-64c9-4b2e-88d0-112c3a3fa2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:55.844774 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d799d85-64c9-4b2e-88d0-112c3a3fa2c4 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:55.845283 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d799d85-64c9-4b2e-88d0-112c3a3fa2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:55.845720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d799d85-64c9-4b2e-88d0-112c3a3fa2c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:55.901518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d182a-9459-4cfd-a57f-9fbeae8cdef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:55.903147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d182a-9459-4cfd-a57f-9fbeae8cdef8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:55.903538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d182a-9459-4cfd-a57f-9fbeae8cdef8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:55.903967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d182a-9459-4cfd-a57f-9fbeae8cdef8 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:55.904818 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d182a-9459-4cfd-a57f-9fbeae8cdef8 None admin] RBAC: Adding query filter params (name=member) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:55.905223 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d182a-9459-4cfd-a57f-9fbeae8cdef8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:55.907171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d182a-9459-4cfd-a57f-9fbeae8cdef8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:55.917834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 119/257] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:55.924340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d182a-9459-4cfd-a57f-9fbeae8cdef8 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:55.970607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d799d85-64c9-4b2e-88d0-112c3a3fa2c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:55.972093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d799d85-64c9-4b2e-88d0-112c3a3fa2c4 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:56.096620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d182a-9459-4cfd-a57f-9fbeae8cdef8 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:56.097289 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:56.163619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 139/258] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:56.165870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d799d85-64c9-4b2e-88d0-112c3a3fa2c4 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:56.166875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d4a4f-17dc-4ad7-8144-0851c391bbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:56.167517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d4a4f-17dc-4ad7-8144-0851c391bbd5 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:56.167803 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d4a4f-17dc-4ad7-8144-0851c391bbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:56.168078 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d4a4f-17dc-4ad7-8144-0851c391bbd5 None admin] PATH_INFO: `/v3/users/40ffc95ccbce4d12b0db524b56bc5436` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:56.173875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d4a4f-17dc-4ad7-8144-0851c391bbd5 None admin] RBAC: Authorizing `identity:get_user(user_id=40ffc95ccbce4d12b0db524b56bc5436)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:56.175361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d4a4f-17dc-4ad7-8144-0851c391bbd5 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:56.177231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 120/259] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:55 2022] GET /identity/v3/users/40ffc95ccbce4d12b0db524b56bc5436 => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:56.178015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d4a4f-17dc-4ad7-8144-0851c391bbd5 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:56.221353 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:56.325392 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d799d85-64c9-4b2e-88d0-112c3a3fa2c4 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:56.338011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d4a4f-17dc-4ad7-8144-0851c391bbd5 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:56.377683 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64a718-1e7f-400a-8dcb-19a02ff6f9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:56.378318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64a718-1e7f-400a-8dcb-19a02ff6f9f1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:56.378632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64a718-1e7f-400a-8dcb-19a02ff6f9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:56.378910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64a718-1e7f-400a-8dcb-19a02ff6f9f1 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:56.380044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64a718-1e7f-400a-8dcb-19a02ff6f9f1 None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:56.382055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64a718-1e7f-400a-8dcb-19a02ff6f9f1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:56.384500 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 121/260] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:56 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:56.385047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64a718-1e7f-400a-8dcb-19a02ff6f9f1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:56.406800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aff88d-5748-493a-8fb3-c0130633e5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:56.407432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aff88d-5748-493a-8fb3-c0130633e5ea None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:56.407708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aff88d-5748-493a-8fb3-c0130633e5ea None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:56.407975 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aff88d-5748-493a-8fb3-c0130633e5ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:56.430130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aff88d-5748-493a-8fb3-c0130633e5ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-a7aff88d-5748-493a-8fb3-c0130633e5ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:56.436674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:56.437670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 140/261] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:56.438888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aff88d-5748-493a-8fb3-c0130633e5ea None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:56.482041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:56.564554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:56.580766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64a718-1e7f-400a-8dcb-19a02ff6f9f1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:56.622345 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aff88d-5748-493a-8fb3-c0130633e5ea None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:56.638836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3890338-ee39-4644-b75b-70737dfcd062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:56.639470 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3890338-ee39-4644-b75b-70737dfcd062 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:56.639746 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3890338-ee39-4644-b75b-70737dfcd062 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:56.640018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3890338-ee39-4644-b75b-70737dfcd062 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:56.642199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3890338-ee39-4644-b75b-70737dfcd062 None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:56.643652 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3890338-ee39-4644-b75b-70737dfcd062 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:56.645270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 122/262] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:56 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:56.646016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3890338-ee39-4644-b75b-70737dfcd062 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:56.683772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a737437-4621-4026-8e17-b37a2b9f406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:56.684626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a737437-4621-4026-8e17-b37a2b9f406b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:56.684901 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a737437-4621-4026-8e17-b37a2b9f406b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:56.690140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a737437-4621-4026-8e17-b37a2b9f406b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:56.690140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a737437-4621-4026-8e17-b37a2b9f406b None admin] RBAC: Adding query filter params (role.id=96271a2245c144d4b43dea66f18074e8, scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30, user.id=6a69eddf36e84f3e9376277156d7e835) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:56.690140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a737437-4621-4026-8e17-b37a2b9f406b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:56.690140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a737437-4621-4026-8e17-b37a2b9f406b None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:56.699680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 141/263] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:56 2022] GET /identity/v3/role_assignments?user.id=6a69eddf36e84f3e9376277156d7e835&scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30&role.id=96271a2245c144d4b43dea66f18074e8 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:56.700258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a737437-4621-4026-8e17-b37a2b9f406b None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:56.838986 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3890338-ee39-4644-b75b-70737dfcd062 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:56.858036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:56.887185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a737437-4621-4026-8e17-b37a2b9f406b None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:56.903328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d7e64-3373-4660-89ce-8b5fa0687036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:56.903956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d7e64-3373-4660-89ce-8b5fa0687036 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:56.904231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d7e64-3373-4660-89ce-8b5fa0687036 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:56.904500 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d7e64-3373-4660-89ce-8b5fa0687036 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:56.905166 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d7e64-3373-4660-89ce-8b5fa0687036 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:56.905463 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d7e64-3373-4660-89ce-8b5fa0687036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:56.906856 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d7e64-3373-4660-89ce-8b5fa0687036 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:56.921222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 123/264] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:56.927473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d7e64-3373-4660-89ce-8b5fa0687036 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:56.956368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b307647-a15d-44f8-b546-91a9a7b1add9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:56.957045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b307647-a15d-44f8-b546-91a9a7b1add9 None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:56.957317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b307647-a15d-44f8-b546-91a9a7b1add9 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:56.957585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b307647-a15d-44f8-b546-91a9a7b1add9 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30/users/40ffc95ccbce4d12b0db524b56bc5436/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:56.962243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b307647-a15d-44f8-b546-91a9a7b1add9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3cab176a56624fcbb4fd9cf5ad72ab30, user_id=40ffc95ccbce4d12b0db524b56bc5436, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:56.963730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b307647-a15d-44f8-b546-91a9a7b1add9 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:56.970851 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:57.058441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 142/265] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:36:56 2022] PUT /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30/users/40ffc95ccbce4d12b0db524b56bc5436/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:36:57.102021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d7e64-3373-4660-89ce-8b5fa0687036 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:57.164951 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0359cc9-0b6c-4aae-974b-ac70acfa3c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:57.165607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0359cc9-0b6c-4aae-974b-ac70acfa3c8e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:57.165881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0359cc9-0b6c-4aae-974b-ac70acfa3c8e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:57.166152 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0359cc9-0b6c-4aae-974b-ac70acfa3c8e None admin] PATH_INFO: `/v3/users/6a69eddf36e84f3e9376277156d7e835` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:57.167382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0359cc9-0b6c-4aae-974b-ac70acfa3c8e None admin] RBAC: Authorizing `identity:get_user(user_id=6a69eddf36e84f3e9376277156d7e835)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:57.168920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0359cc9-0b6c-4aae-974b-ac70acfa3c8e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:57.170947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 124/266] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:56 2022] GET /identity/v3/users/6a69eddf36e84f3e9376277156d7e835 => generated 283 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:57.176310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b307647-a15d-44f8-b546-91a9a7b1add9 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:57.342509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b307647-a15d-44f8-b546-91a9a7b1add9 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:57.379853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbd138-7a4f-4352-ad0e-f2105d74a004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:57.380416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbd138-7a4f-4352-ad0e-f2105d74a004 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:57.380693 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbd138-7a4f-4352-ad0e-f2105d74a004 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:57.380959 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbd138-7a4f-4352-ad0e-f2105d74a004 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:57.397615 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:57.480951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbd138-7a4f-4352-ad0e-f2105d74a004 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:57.482424 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbd138-7a4f-4352-ad0e-f2105d74a004 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:57.645596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 143/267] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:36:57.648023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9d816-6c4f-49e4-8c43-55ee6d2b7152 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:57.649198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9d816-6c4f-49e4-8c43-55ee6d2b7152 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:57.649198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9d816-6c4f-49e4-8c43-55ee6d2b7152 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:57.658869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 125/268] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:57 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:57.659715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbd138-7a4f-4352-ad0e-f2105d74a004 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:57.660007 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b222e02-ddd1-46f8-9ad8-bba15920bc7f None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:57.660288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b222e02-ddd1-46f8-9ad8-bba15920bc7f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:57.660555 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b222e02-ddd1-46f8-9ad8-bba15920bc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:57.705130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-3b222e02-ddd1-46f8-9ad8-bba15920bc7f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:57.706817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3b222e02-ddd1-46f8-9ad8-bba15920bc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:57.771223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:57.888946 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 126/269] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:57.928565 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbd138-7a4f-4352-ad0e-f2105d74a004 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:57.988252 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b7463-759b-4c75-9d66-5b95f920f2df None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:57.988629 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b7463-759b-4c75-9d66-5b95f920f2df None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:57.989179 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b7463-759b-4c75-9d66-5b95f920f2df None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:57.995699 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 127/270] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:58.009147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8cde8-dc67-4ddc-81bc-a7cb905e3055 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:58.009659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8cde8-dc67-4ddc-81bc-a7cb905e3055 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:58.010090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8cde8-dc67-4ddc-81bc-a7cb905e3055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:58.035260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcab79-af0f-48df-aed3-e5202b630781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:58.035915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcab79-af0f-48df-aed3-e5202b630781 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:58.038511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcab79-af0f-48df-aed3-e5202b630781 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:58.038511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcab79-af0f-48df-aed3-e5202b630781 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:58.042513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcab79-af0f-48df-aed3-e5202b630781 None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:58.042513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcab79-af0f-48df-aed3-e5202b630781 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:58.046529 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 144/271] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:57 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:58.058510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcab79-af0f-48df-aed3-e5202b630781 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:58.062512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-3cd8cde8-dc67-4ddc-81bc-a7cb905e3055 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:58.064344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3cd8cde8-dc67-4ddc-81bc-a7cb905e3055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:58.114950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:58.219550 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 128/272] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:58.220227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8cde8-dc67-4ddc-81bc-a7cb905e3055 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:58.279605 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcab79-af0f-48df-aed3-e5202b630781 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:58.305608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:58.319197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77c680-bfa7-4670-8353-d7733d988e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:58.320141 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77c680-bfa7-4670-8353-d7733d988e19 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:58.320693 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77c680-bfa7-4670-8353-d7733d988e19 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:58.321490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77c680-bfa7-4670-8353-d7733d988e19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:58.322730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77c680-bfa7-4670-8353-d7733d988e19 None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30, user.id=6a69eddf36e84f3e9376277156d7e835) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:58.323351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77c680-bfa7-4670-8353-d7733d988e19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:58.324868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77c680-bfa7-4670-8353-d7733d988e19 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:58.325077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8cde8-dc67-4ddc-81bc-a7cb905e3055 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:58.332342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 145/273] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:36:58 2022] GET /identity/v3/role_assignments?user.id=6a69eddf36e84f3e9376277156d7e835&scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:58.333941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30393bc-66c2-4b7e-b513-1dd336be0725 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:58.334297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30393bc-66c2-4b7e-b513-1dd336be0725 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:58.334664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30393bc-66c2-4b7e-b513-1dd336be0725 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:58.338029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 146/274] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:58.343543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ddd47-67f8-445e-86f5-b48ee630101d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:58.344137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ddd47-67f8-445e-86f5-b48ee630101d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:58.344137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ddd47-67f8-445e-86f5-b48ee630101d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:58.380844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-881ddd47-67f8-445e-86f5-b48ee630101d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:58.382069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-881ddd47-67f8-445e-86f5-b48ee630101d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:58.386746 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089cdf9-41a2-4ec3-8511-bc61d6e42c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:58.387405 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089cdf9-41a2-4ec3-8511-bc61d6e42c34 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:58.387735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089cdf9-41a2-4ec3-8511-bc61d6e42c34 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:58.388005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089cdf9-41a2-4ec3-8511-bc61d6e42c34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:58.410044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089cdf9-41a2-4ec3-8511-bc61d6e42c34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-3089cdf9-41a2-4ec3-8511-bc61d6e42c34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:58.419828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:58.422254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:58.424156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:58.424156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:58.424156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:58.424156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:58.424156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:58.424156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:58.424156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:58.424156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:58.424156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:58.424156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:58.424156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:36:58.424156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:36:58.424156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 129/275] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:58.430513 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089cdf9-41a2-4ec3-8511-bc61d6e42c34 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:58.456621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:58.515172 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 147/276] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:36:58.576573 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ddd47-67f8-445e-86f5-b48ee630101d None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:58.607088 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089cdf9-41a2-4ec3-8511-bc61d6e42c34 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:58.643249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c018e-bca8-4a65-9f61-aff2340ccbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:58.644229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c018e-bca8-4a65-9f61-aff2340ccbd8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:58.644647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c018e-bca8-4a65-9f61-aff2340ccbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:58.645179 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c018e-bca8-4a65-9f61-aff2340ccbd8 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:58.646278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c018e-bca8-4a65-9f61-aff2340ccbd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:58.646984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c018e-bca8-4a65-9f61-aff2340ccbd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:58.649324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c018e-bca8-4a65-9f61-aff2340ccbd8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:58.661292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 130/277] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:58.667330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c018e-bca8-4a65-9f61-aff2340ccbd8 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:58.743127 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ddd47-67f8-445e-86f5-b48ee630101d None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:58.752571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:58.764495 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:58.779663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc754f-e55a-4b89-8721-18d5a304b289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:58.780612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc754f-e55a-4b89-8721-18d5a304b289 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:58.781081 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc754f-e55a-4b89-8721-18d5a304b289 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:58.781537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc754f-e55a-4b89-8721-18d5a304b289 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:58.782854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc754f-e55a-4b89-8721-18d5a304b289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:58.784531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc754f-e55a-4b89-8721-18d5a304b289 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:58.786568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 148/278] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:36:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:58.792358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc754f-e55a-4b89-8721-18d5a304b289 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:58.798590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c018e-bca8-4a65-9f61-aff2340ccbd8 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:58.834768 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d7432-113f-4310-b3a1-49bfad34ad28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:58.835398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d7432-113f-4310-b3a1-49bfad34ad28 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:58.835685 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d7432-113f-4310-b3a1-49bfad34ad28 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:58.836513 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d7432-113f-4310-b3a1-49bfad34ad28 None admin] PATH_INFO: `/v3/users/b3d17e25e38b49fc94c92b52f329bfed` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:58.854842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d7432-113f-4310-b3a1-49bfad34ad28 None admin] RBAC: Authorizing `identity:get_user(user_id=b3d17e25e38b49fc94c92b52f329bfed)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:58.856378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d7432-113f-4310-b3a1-49bfad34ad28 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:58.858230 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 131/279] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:58 2022] GET /identity/v3/users/b3d17e25e38b49fc94c92b52f329bfed => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:58.859027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d7432-113f-4310-b3a1-49bfad34ad28 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:58.907808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc754f-e55a-4b89-8721-18d5a304b289 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:58.944787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bbb14-a5f5-4acf-8014-263529999cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:58.946093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bbb14-a5f5-4acf-8014-263529999cca None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:58.946808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bbb14-a5f5-4acf-8014-263529999cca None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:58.947106 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bbb14-a5f5-4acf-8014-263529999cca None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:58.947868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bbb14-a5f5-4acf-8014-263529999cca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:58.949296 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bbb14-a5f5-4acf-8014-263529999cca None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:58.955673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d7432-113f-4310-b3a1-49bfad34ad28 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:58.981444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:58.988980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a957ae2-2559-481c-bfc5-faff62c0c83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:58.989806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a957ae2-2559-481c-bfc5-faff62c0c83b None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:58.990097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a957ae2-2559-481c-bfc5-faff62c0c83b None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:58.990342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a957ae2-2559-481c-bfc5-faff62c0c83b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:59.001602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a957ae2-2559-481c-bfc5-faff62c0c83b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-1a957ae2-2559-481c-bfc5-faff62c0c83b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:36:59.006342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:36:59.007302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 132/280] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:36:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:36:59.009135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a957ae2-2559-481c-bfc5-faff62c0c83b None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:59.046743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 149/281] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:36:58 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:36:59.047878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bbb14-a5f5-4acf-8014-263529999cca None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:59.093453 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:59.145473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a957ae2-2559-481c-bfc5-faff62c0c83b None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:59.198812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b722f-9a1b-41c8-af5b-6fe9f1bbf439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:59.199890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b722f-9a1b-41c8-af5b-6fe9f1bbf439 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:59.200421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b722f-9a1b-41c8-af5b-6fe9f1bbf439 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:59.200700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b722f-9a1b-41c8-af5b-6fe9f1bbf439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:59.201441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b722f-9a1b-41c8-af5b-6fe9f1bbf439 None admin] RBAC: Adding query filter params (role.id=43667d074eda4143bf3b7f721e83810c, user.id=b3d17e25e38b49fc94c92b52f329bfed, scope.system=all) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:59.201748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b722f-9a1b-41c8-af5b-6fe9f1bbf439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:59.204134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b722f-9a1b-41c8-af5b-6fe9f1bbf439 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:59.210743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bbb14-a5f5-4acf-8014-263529999cca None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:59.218522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 133/282] 10.222.0.46 () {56 vars in 1308 bytes} [Fri Sep 9 12:36:59 2022] GET /identity/v3/role_assignments?user.id=b3d17e25e38b49fc94c92b52f329bfed&scope.system=all&role.id=43667d074eda4143bf3b7f721e83810c => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:59.254163 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985c3be-6dce-4fb2-9e8e-7a4452cb8f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:59.254819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985c3be-6dce-4fb2-9e8e-7a4452cb8f01 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:59.255091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985c3be-6dce-4fb2-9e8e-7a4452cb8f01 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:59.255357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985c3be-6dce-4fb2-9e8e-7a4452cb8f01 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:59.256034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985c3be-6dce-4fb2-9e8e-7a4452cb8f01 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:36:59.256331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985c3be-6dce-4fb2-9e8e-7a4452cb8f01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:59.258229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985c3be-6dce-4fb2-9e8e-7a4452cb8f01 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:59.268011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 150/283] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:36:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:59.273650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b722f-9a1b-41c8-af5b-6fe9f1bbf439 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:59.338833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:59.392095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b722f-9a1b-41c8-af5b-6fe9f1bbf439 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:59.433555 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea504a-c93b-4a94-9963-87f70e66ff46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:59.434457 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea504a-c93b-4a94-9963-87f70e66ff46 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:59.434755 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea504a-c93b-4a94-9963-87f70e66ff46 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:59.435026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea504a-c93b-4a94-9963-87f70e66ff46 None admin] PATH_INFO: `/v3/users/991b210436144ad5b367071555b51902` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:59.436343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea504a-c93b-4a94-9963-87f70e66ff46 None admin] RBAC: Authorizing `identity:get_user(user_id=991b210436144ad5b367071555b51902)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:59.437774 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea504a-c93b-4a94-9963-87f70e66ff46 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:59.439507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 134/284] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:36:59 2022] GET /identity/v3/users/991b210436144ad5b367071555b51902 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:36:59.445028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985c3be-6dce-4fb2-9e8e-7a4452cb8f01 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:59.547424 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:59.556754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985c3be-6dce-4fb2-9e8e-7a4452cb8f01 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:59.594731 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5075e8-9fcc-4bd3-9eb7-5ec1798b3796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:59.595688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5075e8-9fcc-4bd3-9eb7-5ec1798b3796 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:59.596154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5075e8-9fcc-4bd3-9eb7-5ec1798b3796 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:59.596633 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5075e8-9fcc-4bd3-9eb7-5ec1798b3796 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:59.597936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5075e8-9fcc-4bd3-9eb7-5ec1798b3796 None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:59.599592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5075e8-9fcc-4bd3-9eb7-5ec1798b3796 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:59.601726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 151/285] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:36:59 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:36:59.607518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea504a-c93b-4a94-9963-87f70e66ff46 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:36:59.717675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea504a-c93b-4a94-9963-87f70e66ff46 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:36:59.741607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:36:59.758839 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8ce5a-a936-44a6-9273-d01f359f2ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:36:59.759824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8ce5a-a936-44a6-9273-d01f359f2ef5 None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:59.760304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8ce5a-a936-44a6-9273-d01f359f2ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:59.760793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8ce5a-a936-44a6-9273-d01f359f2ef5 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30/users/991b210436144ad5b367071555b51902/roles/ca39d64092424a34aaaf4818ca16e988` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:59.763135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8ce5a-a936-44a6-9273-d01f359f2ef5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3cab176a56624fcbb4fd9cf5ad72ab30, user_id=991b210436144ad5b367071555b51902, role_id=ca39d64092424a34aaaf4818ca16e988)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:36:59.764824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8ce5a-a936-44a6-9273-d01f359f2ef5 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:36:59.863150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 135/286] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:36:59 2022] PUT /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30/users/991b210436144ad5b367071555b51902/roles/ca39d64092424a34aaaf4818ca16e988 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:36:59.865863 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b7446-4f0c-4bc4-a9d5-52fcdb070c15 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:59.866143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b7446-4f0c-4bc4-a9d5-52fcdb070c15 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:59.866418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b7446-4f0c-4bc4-a9d5-52fcdb070c15 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:59.868694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 152/287] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:36:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:36:59.874739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e2ecd-ad24-4f68-8c2f-4eca9230a555 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:36:59.875384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e2ecd-ad24-4f68-8c2f-4eca9230a555 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:36:59.875846 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e2ecd-ad24-4f68-8c2f-4eca9230a555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:36:59.899186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-734e2ecd-ad24-4f68-8c2f-4eca9230a555 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:36:59.900757 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-734e2ecd-ad24-4f68-8c2f-4eca9230a555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:36:59.981349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 136/288] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:36:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:00.292616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b7446-4f0c-4bc4-a9d5-52fcdb070c15 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:00.415492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b7446-4f0c-4bc4-a9d5-52fcdb070c15 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:00.452228 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d0be2-711f-419c-84ce-fbeeccd811dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:00.452841 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d0be2-711f-419c-84ce-fbeeccd811dd None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:00.453114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d0be2-711f-419c-84ce-fbeeccd811dd None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:00.453384 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d0be2-711f-419c-84ce-fbeeccd811dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:00.464691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d0be2-711f-419c-84ce-fbeeccd811dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-814d0be2-711f-419c-84ce-fbeeccd811dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:00.470616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:00.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:00.472300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:00.472300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:00.472300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:00.472300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:00.472300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:00.472300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:00.472300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:00.472300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:00.472300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:00.472300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:00.472300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:00.472300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:00.472300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 153/289] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:00.477918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e2ecd-ad24-4f68-8c2f-4eca9230a555 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:00.597922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e2ecd-ad24-4f68-8c2f-4eca9230a555 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:00.615945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:00.640717 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e54550-8fc8-47bf-a2f8-4935363254f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:00.641321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e54550-8fc8-47bf-a2f8-4935363254f2 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:00.641595 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e54550-8fc8-47bf-a2f8-4935363254f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:00.641862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e54550-8fc8-47bf-a2f8-4935363254f2 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:00.642538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e54550-8fc8-47bf-a2f8-4935363254f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:00.642839 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e54550-8fc8-47bf-a2f8-4935363254f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:00.644190 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e54550-8fc8-47bf-a2f8-4935363254f2 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:00.652500 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 137/290] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:37:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:00.658310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d0be2-711f-419c-84ce-fbeeccd811dd None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:00.681882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:00.903675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d0be2-711f-419c-84ce-fbeeccd811dd None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:00.942091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0439be-4cee-4128-9de5-8822c48c9945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:00.942670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0439be-4cee-4128-9de5-8822c48c9945 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:00.942960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0439be-4cee-4128-9de5-8822c48c9945 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:00.943258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0439be-4cee-4128-9de5-8822c48c9945 None admin] PATH_INFO: `/v3/users/40ffc95ccbce4d12b0db524b56bc5436` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:00.947173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0439be-4cee-4128-9de5-8822c48c9945 None admin] RBAC: Authorizing `identity:get_user(user_id=40ffc95ccbce4d12b0db524b56bc5436)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:00.948601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0439be-4cee-4128-9de5-8822c48c9945 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:00.950339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 154/291] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:37:00 2022] GET /identity/v3/users/40ffc95ccbce4d12b0db524b56bc5436 => generated 297 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:00.955757 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e54550-8fc8-47bf-a2f8-4935363254f2 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:01.080990 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e54550-8fc8-47bf-a2f8-4935363254f2 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:01.134828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b45a0-b61f-4019-9231-f995e31fede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:01.135438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b45a0-b61f-4019-9231-f995e31fede9 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:01.135719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b45a0-b61f-4019-9231-f995e31fede9 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:01.135992 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b45a0-b61f-4019-9231-f995e31fede9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:01.190678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:01.227149 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b45a0-b61f-4019-9231-f995e31fede9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:01.229620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b45a0-b61f-4019-9231-f995e31fede9 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:01.357358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 138/292] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:01.362910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0439be-4cee-4128-9de5-8822c48c9945 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:01.474856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:01.494518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0439be-4cee-4128-9de5-8822c48c9945 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:01.543396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ad424-dbd8-4f97-8a1f-cf89d0596cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:01.544411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ad424-dbd8-4f97-8a1f-cf89d0596cea None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:01.544881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ad424-dbd8-4f97-8a1f-cf89d0596cea None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:01.545357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ad424-dbd8-4f97-8a1f-cf89d0596cea None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:01.546689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ad424-dbd8-4f97-8a1f-cf89d0596cea None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:01.548300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ad424-dbd8-4f97-8a1f-cf89d0596cea None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:01.550437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 155/293] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:01 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:01.558507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b45a0-b61f-4019-9231-f995e31fede9 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:01.675863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:01.712045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b45a0-b61f-4019-9231-f995e31fede9 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:01.764662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db544ea3-fee5-4999-8c5f-9c1da5ff1c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:01.765642 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db544ea3-fee5-4999-8c5f-9c1da5ff1c6c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:01.766135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db544ea3-fee5-4999-8c5f-9c1da5ff1c6c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:01.766624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db544ea3-fee5-4999-8c5f-9c1da5ff1c6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:01.767568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db544ea3-fee5-4999-8c5f-9c1da5ff1c6c None admin] RBAC: Adding query filter params (role.id=43667d074eda4143bf3b7f721e83810c, scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30, user.id=40ffc95ccbce4d12b0db524b56bc5436) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:01.768079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db544ea3-fee5-4999-8c5f-9c1da5ff1c6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:01.778611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db544ea3-fee5-4999-8c5f-9c1da5ff1c6c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:01.782195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 139/294] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:01 2022] GET /identity/v3/role_assignments?user.id=40ffc95ccbce4d12b0db524b56bc5436&scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30&role.id=43667d074eda4143bf3b7f721e83810c => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:01.856393 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca23e97-2427-46be-a920-0d3ad8d4ca62 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:01.857017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca23e97-2427-46be-a920-0d3ad8d4ca62 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:01.857485 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca23e97-2427-46be-a920-0d3ad8d4ca62 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:01.859123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 156/295] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:01.865208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf23846-3805-4ffe-80e2-90395c8b07e2 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:01.865516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf23846-3805-4ffe-80e2-90395c8b07e2 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:01.865846 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf23846-3805-4ffe-80e2-90395c8b07e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:01.895749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-2bf23846-3805-4ffe-80e2-90395c8b07e2 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:01.897073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2bf23846-3805-4ffe-80e2-90395c8b07e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:01.904822 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda735f-16f9-44d2-a3cc-04d497bd3b4c None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:01.905737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda735f-16f9-44d2-a3cc-04d497bd3b4c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:01.906406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda735f-16f9-44d2-a3cc-04d497bd3b4c None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:01.909831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 157/296] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:01.918894 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3511baa-69ce-46dc-b550-81c83a42fe0f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:01.919534 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3511baa-69ce-46dc-b550-81c83a42fe0f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:01.920006 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3511baa-69ce-46dc-b550-81c83a42fe0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:01.954637 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d3511baa-69ce-46dc-b550-81c83a42fe0f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:01.963866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d3511baa-69ce-46dc-b550-81c83a42fe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:02.007522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 140/297] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:02.080420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 158/298] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:02.155588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab0e1f-175b-4f65-96e3-2322842addaf None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:02.155926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab0e1f-175b-4f65-96e3-2322842addaf None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:02.156135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab0e1f-175b-4f65-96e3-2322842addaf None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:02.157609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 141/299] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:02.162781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab585d31-0dbf-4701-85f5-fcbced46b869 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:02.163066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab585d31-0dbf-4701-85f5-fcbced46b869 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:02.163338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab585d31-0dbf-4701-85f5-fcbced46b869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:02.182689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-ab585d31-0dbf-4701-85f5-fcbced46b869 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:02.183850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ab585d31-0dbf-4701-85f5-fcbced46b869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:02.259349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 159/300] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:02.308157 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab0e1f-175b-4f65-96e3-2322842addaf None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:02.377172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:02.417760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab0e1f-175b-4f65-96e3-2322842addaf None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:02.424392 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab585d31-0dbf-4701-85f5-fcbced46b869 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:02.444950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:02.457623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413da5ec-40a9-4bd5-a38c-627beba33037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:02.458260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413da5ec-40a9-4bd5-a38c-627beba33037 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:02.458580 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413da5ec-40a9-4bd5-a38c-627beba33037 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:02.458855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413da5ec-40a9-4bd5-a38c-627beba33037 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:02.469822 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413da5ec-40a9-4bd5-a38c-627beba33037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-413da5ec-40a9-4bd5-a38c-627beba33037 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:02.474692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:02.475589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:02.476950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:02.476950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:02.476950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:02.476950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:02.476950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:02.476950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:02.476950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:02.476950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:02.476950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:02.476950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:02.476950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:02.476950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:02.476950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 142/301] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:02.482186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413da5ec-40a9-4bd5-a38c-627beba33037 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:02.541523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab585d31-0dbf-4701-85f5-fcbced46b869 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:02.587797 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173ded2-2cd6-4d43-b8be-834d4cda971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:02.588455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173ded2-2cd6-4d43-b8be-834d4cda971f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:02.588837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173ded2-2cd6-4d43-b8be-834d4cda971f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:02.589113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173ded2-2cd6-4d43-b8be-834d4cda971f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:02.599231 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173ded2-2cd6-4d43-b8be-834d4cda971f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-6173ded2-2cd6-4d43-b8be-834d4cda971f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:02.604830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:02.605707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:02.606558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:02.606558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:02.606558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:02.606558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:02.606558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:02.606558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:02.606558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:02.606558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:02.606558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:02.606558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:02.606558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:37:02.606558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:02.606558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 160/302] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:37:02.607220 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173ded2-2cd6-4d43-b8be-834d4cda971f None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:02.614024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413da5ec-40a9-4bd5-a38c-627beba33037 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:02.640629 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:02.650391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d411e-d9f1-4e9e-9c43-9d52159505fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:02.651024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d411e-d9f1-4e9e-9c43-9d52159505fa None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:02.651303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d411e-d9f1-4e9e-9c43-9d52159505fa None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:02.651597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d411e-d9f1-4e9e-9c43-9d52159505fa None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:02.652215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d411e-d9f1-4e9e-9c43-9d52159505fa None admin] RBAC: Adding query filter params (name=member) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:02.652617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d411e-d9f1-4e9e-9c43-9d52159505fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:02.653862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d411e-d9f1-4e9e-9c43-9d52159505fa None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:02.662002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 143/303] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:37:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:02.662625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d411e-d9f1-4e9e-9c43-9d52159505fa None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:02.688522 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:02.713183 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173ded2-2cd6-4d43-b8be-834d4cda971f None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:02.751906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b3519-bf47-4dbf-93d8-ada0ebb23b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:02.752446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b3519-bf47-4dbf-93d8-ada0ebb23b21 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:02.752719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b3519-bf47-4dbf-93d8-ada0ebb23b21 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:02.752990 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b3519-bf47-4dbf-93d8-ada0ebb23b21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:02.754098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b3519-bf47-4dbf-93d8-ada0ebb23b21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:02.755569 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b3519-bf47-4dbf-93d8-ada0ebb23b21 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:02.757301 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 161/304] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:02.762413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b3519-bf47-4dbf-93d8-ada0ebb23b21 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:02.772757 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d411e-d9f1-4e9e-9c43-9d52159505fa None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:02.811568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffaa24-78d0-4cd2-a874-32b0e1a6c1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:02.812189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffaa24-78d0-4cd2-a874-32b0e1a6c1a5 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:02.812472 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffaa24-78d0-4cd2-a874-32b0e1a6c1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:02.812749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffaa24-78d0-4cd2-a874-32b0e1a6c1a5 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:02.813415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffaa24-78d0-4cd2-a874-32b0e1a6c1a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:02.813720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffaa24-78d0-4cd2-a874-32b0e1a6c1a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:02.815186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffaa24-78d0-4cd2-a874-32b0e1a6c1a5 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:02.824036 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 144/305] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:37:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:02.827677 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffaa24-78d0-4cd2-a874-32b0e1a6c1a5 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:02.873048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b3519-bf47-4dbf-93d8-ada0ebb23b21 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:02.880446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:02.909899 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297fc98-44c0-4960-87e3-aee4a10d51e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:02.910443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297fc98-44c0-4960-87e3-aee4a10d51e4 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:02.910749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297fc98-44c0-4960-87e3-aee4a10d51e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:02.911028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297fc98-44c0-4960-87e3-aee4a10d51e4 None admin] PATH_INFO: `/v3/users/b3d17e25e38b49fc94c92b52f329bfed` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:02.912127 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297fc98-44c0-4960-87e3-aee4a10d51e4 None admin] RBAC: Authorizing `identity:get_user(user_id=b3d17e25e38b49fc94c92b52f329bfed)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:02.913557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297fc98-44c0-4960-87e3-aee4a10d51e4 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:02.915205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 162/306] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:37:02 2022] GET /identity/v3/users/b3d17e25e38b49fc94c92b52f329bfed => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:02.915952 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297fc98-44c0-4960-87e3-aee4a10d51e4 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:02.929470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:02.936241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffaa24-78d0-4cd2-a874-32b0e1a6c1a5 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:02.969964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154298e5-2cad-4d13-a4ac-f6bba8143e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:02.970591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154298e5-2cad-4d13-a4ac-f6bba8143e84 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:02.970869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154298e5-2cad-4d13-a4ac-f6bba8143e84 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:02.971137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154298e5-2cad-4d13-a4ac-f6bba8143e84 None admin] PATH_INFO: `/v3/groups` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:02.971754 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154298e5-2cad-4d13-a4ac-f6bba8143e84 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:02.973182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154298e5-2cad-4d13-a4ac-f6bba8143e84 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:03.011383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297fc98-44c0-4960-87e3-aee4a10d51e4 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:03.043821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6b2a1-58d0-4abb-b490-502a24bbdab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:03.044436 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6b2a1-58d0-4abb-b490-502a24bbdab4 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:03.044712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6b2a1-58d0-4abb-b490-502a24bbdab4 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:03.044979 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6b2a1-58d0-4abb-b490-502a24bbdab4 None admin] PATH_INFO: `/v3/users/3f96c267d5ae4563847bcbe8d26ddfad` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:03.057882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6b2a1-58d0-4abb-b490-502a24bbdab4 None admin] RBAC: Authorizing `identity:get_user(user_id=3f96c267d5ae4563847bcbe8d26ddfad)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:03.058007 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 145/307] 10.222.0.46 () {60 vars in 1134 bytes} [Fri Sep 9 12:37:02 2022] POST /identity/v3/groups => generated 229 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:37:03.059442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3253b6-6243-4cdc-9a50-e233c9d67c91 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:03.059692 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6b2a1-58d0-4abb-b490-502a24bbdab4 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:03.059787 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3253b6-6243-4cdc-9a50-e233c9d67c91 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:03.060048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3253b6-6243-4cdc-9a50-e233c9d67c91 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:03.061684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 163/308] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:37:02 2022] GET /identity/v3/users/3f96c267d5ae4563847bcbe8d26ddfad => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:03.061810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 146/309] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:03.066589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6b2a1-58d0-4abb-b490-502a24bbdab4 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:03.072358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e46384-79ef-443b-b5b1-43b7954157be None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:03.072358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e46384-79ef-443b-b5b1-43b7954157be None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:03.072358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e46384-79ef-443b-b5b1-43b7954157be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:03.088926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-77e46384-79ef-443b-b5b1-43b7954157be None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:03.090138 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-77e46384-79ef-443b-b5b1-43b7954157be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:03.153767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:03.178600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 147/310] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:03.178600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e46384-79ef-443b-b5b1-43b7954157be None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:03.200508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6b2a1-58d0-4abb-b490-502a24bbdab4 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:03.224437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:03.253187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7485b-4953-43b4-8b2d-70725552c3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:03.254001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7485b-4953-43b4-8b2d-70725552c3ec None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:03.254276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7485b-4953-43b4-8b2d-70725552c3ec None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:03.254580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7485b-4953-43b4-8b2d-70725552c3ec None admin] PATH_INFO: `/v3/system/users/b3d17e25e38b49fc94c92b52f329bfed/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:03.255989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7485b-4953-43b4-8b2d-70725552c3ec None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b3d17e25e38b49fc94c92b52f329bfed, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:03.257369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7485b-4953-43b4-8b2d-70725552c3ec None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:03.294778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e46384-79ef-443b-b5b1-43b7954157be None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:03.331040 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c92514-a96a-44ec-bc60-d171bb462833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:03.331701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c92514-a96a-44ec-bc60-d171bb462833 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:03.332039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c92514-a96a-44ec-bc60-d171bb462833 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:03.332313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c92514-a96a-44ec-bc60-d171bb462833 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:03.333266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 164/311] 10.222.0.46 () {58 vars in 1334 bytes} [Fri Sep 9 12:37:03 2022] PUT /identity/v3/system/users/b3d17e25e38b49fc94c92b52f329bfed/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:37:03.333392 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c92514-a96a-44ec-bc60-d171bb462833 None admin] RBAC: Adding query filter params (role.id=ca39d64092424a34aaaf4818ca16e988, user.id=3f96c267d5ae4563847bcbe8d26ddfad, scope.system=all) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:03.333469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c92514-a96a-44ec-bc60-d171bb462833 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:03.335042 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c92514-a96a-44ec-bc60-d171bb462833 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:03.343128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 148/312] 10.222.0.46 () {56 vars in 1308 bytes} [Fri Sep 9 12:37:03 2022] GET /identity/v3/role_assignments?user.id=3f96c267d5ae4563847bcbe8d26ddfad&scope.system=all&role.id=ca39d64092424a34aaaf4818ca16e988 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:03.460319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7485b-4953-43b4-8b2d-70725552c3ec None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:03.577005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7485b-4953-43b4-8b2d-70725552c3ec None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:03.611199 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7de895-3726-4fb3-9ab6-023fde11d697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:03.611889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7de895-3726-4fb3-9ab6-023fde11d697 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:03.612168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7de895-3726-4fb3-9ab6-023fde11d697 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:03.612491 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7de895-3726-4fb3-9ab6-023fde11d697 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:03.614356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:03.624828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7de895-3726-4fb3-9ab6-023fde11d697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-1a7de895-3726-4fb3-9ab6-023fde11d697 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:03.629671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:03.630594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:03.631401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:03.631401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:03.631401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:03.631401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:03.631401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:03.631401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:03.631401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:03.631401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:03.631401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:03.631401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:03.631401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:37:03.631401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:03.631401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 165/313] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:03.637243 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c92514-a96a-44ec-bc60-d171bb462833 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:03.743799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c92514-a96a-44ec-bc60-d171bb462833 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:03.746251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:03.784759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed0662-ee25-4b3d-91f5-f06511148038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:03.785726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed0662-ee25-4b3d-91f5-f06511148038 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:03.786016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed0662-ee25-4b3d-91f5-f06511148038 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:03.786288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed0662-ee25-4b3d-91f5-f06511148038 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:03.786988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed0662-ee25-4b3d-91f5-f06511148038 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:03.787290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed0662-ee25-4b3d-91f5-f06511148038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:03.788664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed0662-ee25-4b3d-91f5-f06511148038 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:03.798510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 149/314] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:37:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:03.803754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7de895-3726-4fb3-9ab6-023fde11d697 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:03.907137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7de895-3726-4fb3-9ab6-023fde11d697 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:03.941401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67770c36-c1b1-4e8e-a383-3302c82308c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:03.942034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67770c36-c1b1-4e8e-a383-3302c82308c6 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:03.942297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67770c36-c1b1-4e8e-a383-3302c82308c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:03.942597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67770c36-c1b1-4e8e-a383-3302c82308c6 None admin] PATH_INFO: `/v3/users/991b210436144ad5b367071555b51902` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:03.943718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67770c36-c1b1-4e8e-a383-3302c82308c6 None admin] RBAC: Authorizing `identity:get_user(user_id=991b210436144ad5b367071555b51902)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:03.945664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67770c36-c1b1-4e8e-a383-3302c82308c6 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:03.947397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 166/315] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:37:03 2022] GET /identity/v3/users/991b210436144ad5b367071555b51902 => generated 297 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:03.952999 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed0662-ee25-4b3d-91f5-f06511148038 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:04.063892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed0662-ee25-4b3d-91f5-f06511148038 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:04.098309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f9aa0-f5c8-4f03-b60f-4c4f654f0702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:04.098968 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f9aa0-f5c8-4f03-b60f-4c4f654f0702 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:04.099393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f9aa0-f5c8-4f03-b60f-4c4f654f0702 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:04.099669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f9aa0-f5c8-4f03-b60f-4c4f654f0702 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:04.139224 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:04.173441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f9aa0-f5c8-4f03-b60f-4c4f654f0702 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:04.174931 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f9aa0-f5c8-4f03-b60f-4c4f654f0702 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:04.279140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 150/316] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:04.284640 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67770c36-c1b1-4e8e-a383-3302c82308c6 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:04.352860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:04.394964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67770c36-c1b1-4e8e-a383-3302c82308c6 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:04.429254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d35fd2-a090-4946-ba38-85b97ec65b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:04.429992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d35fd2-a090-4946-ba38-85b97ec65b38 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:04.430264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d35fd2-a090-4946-ba38-85b97ec65b38 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:04.430561 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d35fd2-a090-4946-ba38-85b97ec65b38 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:04.431685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d35fd2-a090-4946-ba38-85b97ec65b38 None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:04.433094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d35fd2-a090-4946-ba38-85b97ec65b38 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:04.435342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 167/317] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:04 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:04.441018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f9aa0-f5c8-4f03-b60f-4c4f654f0702 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:04.507471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:04.542537 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f9aa0-f5c8-4f03-b60f-4c4f654f0702 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:04.578653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81c439-69c7-4341-bc22-baf453428adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:04.579265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81c439-69c7-4341-bc22-baf453428adf None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:04.579546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81c439-69c7-4341-bc22-baf453428adf None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:04.579819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81c439-69c7-4341-bc22-baf453428adf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:04.580542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81c439-69c7-4341-bc22-baf453428adf None admin] RBAC: Adding query filter params (role.id=ca39d64092424a34aaaf4818ca16e988, scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30, user.id=991b210436144ad5b367071555b51902) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:04.580849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81c439-69c7-4341-bc22-baf453428adf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:04.582274 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81c439-69c7-4341-bc22-baf453428adf None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:04.588953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 151/318] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:04 2022] GET /identity/v3/role_assignments?user.id=991b210436144ad5b367071555b51902&scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30&role.id=ca39d64092424a34aaaf4818ca16e988 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:05.346786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197ab31-c32b-4484-a881-d7c3762fec86 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:05.347294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197ab31-c32b-4484-a881-d7c3762fec86 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:05.347368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197ab31-c32b-4484-a881-d7c3762fec86 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:05.348705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 168/319] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:05.354517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295350c5-5fd2-40d4-b16c-c2ae7864320c None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:05.354895 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295350c5-5fd2-40d4-b16c-c2ae7864320c None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:05.355055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295350c5-5fd2-40d4-b16c-c2ae7864320c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:05.373008 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-295350c5-5fd2-40d4-b16c-c2ae7864320c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:05.374124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-295350c5-5fd2-40d4-b16c-c2ae7864320c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:05.444440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 152/320] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:05.587568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614721c-0750-4b56-9d16-7427ff8f98c4 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:05.587978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614721c-0750-4b56-9d16-7427ff8f98c4 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:05.588170 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614721c-0750-4b56-9d16-7427ff8f98c4 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:05.589692 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 169/321] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:05.595488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6d1d7-b56a-4e58-88c2-105c8adc03e0 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:05.595806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6d1d7-b56a-4e58-88c2-105c8adc03e0 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:05.596050 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6d1d7-b56a-4e58-88c2-105c8adc03e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:05.618016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-8ff6d1d7-b56a-4e58-88c2-105c8adc03e0 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:05.621734 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-8ff6d1d7-b56a-4e58-88c2-105c8adc03e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:05.679517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:05.691588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 153/322] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:05.702746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a3cb0-bb7d-4a5b-8671-93112720a39a None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:05.703026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a3cb0-bb7d-4a5b-8671-93112720a39a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:05.703292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a3cb0-bb7d-4a5b-8671-93112720a39a None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:05.704733 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 170/323] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:05.710031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511b8e5-c6ab-42c6-9e75-447ce93c776b None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:05.711305 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511b8e5-c6ab-42c6-9e75-447ce93c776b None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:05.712045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511b8e5-c6ab-42c6-9e75-447ce93c776b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:05.730392 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-7511b8e5-c6ab-42c6-9e75-447ce93c776b None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:05.731592 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7511b8e5-c6ab-42c6-9e75-447ce93c776b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:05.737504 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a3cb0-bb7d-4a5b-8671-93112720a39a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:05.803510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 154/324] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:05.846928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a3cb0-bb7d-4a5b-8671-93112720a39a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:05.874883 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:05.880627 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d96f1a-55d3-4cb0-8e42-5431da73af74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:05.881249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d96f1a-55d3-4cb0-8e42-5431da73af74 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:05.881524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d96f1a-55d3-4cb0-8e42-5431da73af74 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:05.881787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d96f1a-55d3-4cb0-8e42-5431da73af74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:05.883544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d96f1a-55d3-4cb0-8e42-5431da73af74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:05.884942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d96f1a-55d3-4cb0-8e42-5431da73af74 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:05.890591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 171/325] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:05.892137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511b8e5-c6ab-42c6-9e75-447ce93c776b None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:05.969725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d96f1a-55d3-4cb0-8e42-5431da73af74 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:06.006830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511b8e5-c6ab-42c6-9e75-447ce93c776b None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:06.027777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:06.042361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25d965-2d1c-46fb-b5b1-75fe00316766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:06.043270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25d965-2d1c-46fb-b5b1-75fe00316766 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:06.043730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25d965-2d1c-46fb-b5b1-75fe00316766 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:06.043998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25d965-2d1c-46fb-b5b1-75fe00316766 None admin] PATH_INFO: `/v3/groups` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:06.044636 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25d965-2d1c-46fb-b5b1-75fe00316766 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:06.046052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25d965-2d1c-46fb-b5b1-75fe00316766 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:06.075098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d96f1a-55d3-4cb0-8e42-5431da73af74 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:06.104194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 155/326] 10.222.0.46 () {60 vars in 1134 bytes} [Fri Sep 9 12:37:05 2022] POST /identity/v3/groups => generated 226 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:37:06.105447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25d965-2d1c-46fb-b5b1-75fe00316766 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:06.126276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d23a3b-8cfc-4f6d-8038-59d839290c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:06.126972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d23a3b-8cfc-4f6d-8038-59d839290c45 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:06.127244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d23a3b-8cfc-4f6d-8038-59d839290c45 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:06.127513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d23a3b-8cfc-4f6d-8038-59d839290c45 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:06.140454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d23a3b-8cfc-4f6d-8038-59d839290c45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-d3d23a3b-8cfc-4f6d-8038-59d839290c45 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:06.145911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:06.147770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:06.149613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:06.149613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:06.149613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:06.149613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:06.149613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:06.149613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:06.149613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:06.149613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:06.149613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:06.149613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:06.149613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:37:06.149613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:06.149613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 172/327] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:06.156463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d23a3b-8cfc-4f6d-8038-59d839290c45 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:06.221543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:06.235624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25d965-2d1c-46fb-b5b1-75fe00316766 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:06.267476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d23a3b-8cfc-4f6d-8038-59d839290c45 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:06.274753 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5606afc-eb3c-4022-8914-579e46ba67a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:06.275563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5606afc-eb3c-4022-8914-579e46ba67a9 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:06.275853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5606afc-eb3c-4022-8914-579e46ba67a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:06.276128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5606afc-eb3c-4022-8914-579e46ba67a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:06.299417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5606afc-eb3c-4022-8914-579e46ba67a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-d5606afc-eb3c-4022-8914-579e46ba67a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:06.308532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:06.309910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:06.310935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:06.310935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:06.310935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:06.310935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:06.310935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:06.310935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:06.310935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:06.310935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:06.310935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:06.310935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:06.310935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:06.310935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:06.310935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 156/328] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:06.316601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5606afc-eb3c-4022-8914-579e46ba67a9 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:06.321399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642cb06-7c7b-4734-b01e-f432b682b33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:06.322078 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642cb06-7c7b-4734-b01e-f432b682b33e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:06.322364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642cb06-7c7b-4734-b01e-f432b682b33e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:06.322668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642cb06-7c7b-4734-b01e-f432b682b33e None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:06.323767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642cb06-7c7b-4734-b01e-f432b682b33e None admin] RBAC: Adding query filter params (name=reader) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:06.323767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642cb06-7c7b-4734-b01e-f432b682b33e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:06.325015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642cb06-7c7b-4734-b01e-f432b682b33e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:06.335360 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 173/329] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:37:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:06.341318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642cb06-7c7b-4734-b01e-f432b682b33e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:06.369698 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:06.450165 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5606afc-eb3c-4022-8914-579e46ba67a9 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:06.463908 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642cb06-7c7b-4734-b01e-f432b682b33e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:06.490164 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:06.494906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575619c-6e03-48b1-920a-6f733aad4459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:06.495694 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575619c-6e03-48b1-920a-6f733aad4459 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:06.496157 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575619c-6e03-48b1-920a-6f733aad4459 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:06.496552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575619c-6e03-48b1-920a-6f733aad4459 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:06.497208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575619c-6e03-48b1-920a-6f733aad4459 None admin] RBAC: Adding query filter params (name=member) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:06.497518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575619c-6e03-48b1-920a-6f733aad4459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:06.498934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575619c-6e03-48b1-920a-6f733aad4459 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:06.506139 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c873fe-941c-4067-ac83-51e8958219a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:06.507057 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c873fe-941c-4067-ac83-51e8958219a8 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:06.507790 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c873fe-941c-4067-ac83-51e8958219a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:06.507790 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c873fe-941c-4067-ac83-51e8958219a8 None admin] PATH_INFO: `/v3/users/3f96c267d5ae4563847bcbe8d26ddfad` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:06.509209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c873fe-941c-4067-ac83-51e8958219a8 None admin] RBAC: Authorizing `identity:get_user(user_id=3f96c267d5ae4563847bcbe8d26ddfad)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:06.510549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 157/330] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:37:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:06.511876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c873fe-941c-4067-ac83-51e8958219a8 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:06.513494 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 174/331] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:37:06 2022] GET /identity/v3/users/3f96c267d5ae4563847bcbe8d26ddfad => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:06.519356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c873fe-941c-4067-ac83-51e8958219a8 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:06.519493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575619c-6e03-48b1-920a-6f733aad4459 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:06.657570 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c873fe-941c-4067-ac83-51e8958219a8 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:06.666954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575619c-6e03-48b1-920a-6f733aad4459 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:06.669752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:06.699122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8014b-2fef-4e3a-b82c-9dac10aba420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:06.700058 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8014b-2fef-4e3a-b82c-9dac10aba420 None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:06.700517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8014b-2fef-4e3a-b82c-9dac10aba420 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:06.700960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8014b-2fef-4e3a-b82c-9dac10aba420 None admin] PATH_INFO: `/v3/system/users/3f96c267d5ae4563847bcbe8d26ddfad/roles/ca39d64092424a34aaaf4818ca16e988` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:06.702699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8014b-2fef-4e3a-b82c-9dac10aba420 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3f96c267d5ae4563847bcbe8d26ddfad, role_id=ca39d64092424a34aaaf4818ca16e988)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:06.704302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8014b-2fef-4e3a-b82c-9dac10aba420 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:06.713459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fbe19-ae37-40f5-83ea-2d7624333eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:06.714206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fbe19-ae37-40f5-83ea-2d7624333eb7 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:06.717060 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fbe19-ae37-40f5-83ea-2d7624333eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:06.717060 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fbe19-ae37-40f5-83ea-2d7624333eb7 None admin] PATH_INFO: `/v3/users/b3d17e25e38b49fc94c92b52f329bfed` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:06.724605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fbe19-ae37-40f5-83ea-2d7624333eb7 None admin] RBAC: Authorizing `identity:get_user(user_id=b3d17e25e38b49fc94c92b52f329bfed)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:06.724605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fbe19-ae37-40f5-83ea-2d7624333eb7 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:06.724605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 158/332] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:37:06 2022] GET /identity/v3/users/b3d17e25e38b49fc94c92b52f329bfed => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:06.727559 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fbe19-ae37-40f5-83ea-2d7624333eb7 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:06.780048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 175/333] 10.222.0.46 () {58 vars in 1334 bytes} [Fri Sep 9 12:37:06 2022] PUT /identity/v3/system/users/3f96c267d5ae4563847bcbe8d26ddfad/roles/ca39d64092424a34aaaf4818ca16e988 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:37:06.843725 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fbe19-ae37-40f5-83ea-2d7624333eb7 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:06.889369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed96fb2-9032-4db3-acf3-fc5673cc6a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:06.889982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed96fb2-9032-4db3-acf3-fc5673cc6a30 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:06.890257 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed96fb2-9032-4db3-acf3-fc5673cc6a30 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:06.890550 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed96fb2-9032-4db3-acf3-fc5673cc6a30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:06.891282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed96fb2-9032-4db3-acf3-fc5673cc6a30 None admin] RBAC: Adding query filter params (role.id=43667d074eda4143bf3b7f721e83810c, user.id=b3d17e25e38b49fc94c92b52f329bfed, scope.system=all) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:06.891584 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed96fb2-9032-4db3-acf3-fc5673cc6a30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:06.893028 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed96fb2-9032-4db3-acf3-fc5673cc6a30 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:06.912066 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 159/334] 10.222.0.46 () {56 vars in 1308 bytes} [Fri Sep 9 12:37:06 2022] GET /identity/v3/role_assignments?user.id=b3d17e25e38b49fc94c92b52f329bfed&scope.system=all&role.id=43667d074eda4143bf3b7f721e83810c => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:08.570403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7686e-9a42-4afe-b0d7-d5a5f0730625 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:08.571447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7686e-9a42-4afe-b0d7-d5a5f0730625 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:08.571933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7686e-9a42-4afe-b0d7-d5a5f0730625 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:08.573800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 176/335] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:08.579240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b0ccb-f02e-43fa-8b48-73fc5ff84544 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:08.580110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b0ccb-f02e-43fa-8b48-73fc5ff84544 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:08.580587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b0ccb-f02e-43fa-8b48-73fc5ff84544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:08.600186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-de7b0ccb-f02e-43fa-8b48-73fc5ff84544 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:08.602277 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-de7b0ccb-f02e-43fa-8b48-73fc5ff84544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:08.630305 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:08.685308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 160/336] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:09.163142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296e038-0877-4a42-973b-b29e3930c0f0 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:09.163836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296e038-0877-4a42-973b-b29e3930c0f0 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:09.164423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296e038-0877-4a42-973b-b29e3930c0f0 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:09.166318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 177/337] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:09.180881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888ef6a-5abb-4f12-b52b-31d2183eac83 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:09.181176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888ef6a-5abb-4f12-b52b-31d2183eac83 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:09.181444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888ef6a-5abb-4f12-b52b-31d2183eac83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:09.215943 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-6888ef6a-5abb-4f12-b52b-31d2183eac83 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:09.217317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6888ef6a-5abb-4f12-b52b-31d2183eac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:09.322862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 161/338] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:09.352628 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296e038-0877-4a42-973b-b29e3930c0f0 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:09.437681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:09.439927 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a85cb-7712-4227-832c-9405ccb0beb5 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:09.440211 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a85cb-7712-4227-832c-9405ccb0beb5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:09.440474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a85cb-7712-4227-832c-9405ccb0beb5 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:09.442160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 162/339] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:09.463842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3f936-a4fe-4a62-aa7a-9c32e9606742 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:09.468400 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3f936-a4fe-4a62-aa7a-9c32e9606742 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:09.468957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3f936-a4fe-4a62-aa7a-9c32e9606742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:09.515667 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-20a3f936-a4fe-4a62-aa7a-9c32e9606742 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:09.517387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-20a3f936-a4fe-4a62-aa7a-9c32e9606742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:09.540135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296e038-0877-4a42-973b-b29e3930c0f0 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:09.603729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4cb93-f581-4667-8815-7efda593148c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:09.605264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4cb93-f581-4667-8815-7efda593148c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:09.605794 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4cb93-f581-4667-8815-7efda593148c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:09.606263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4cb93-f581-4667-8815-7efda593148c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:09.620911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4cb93-f581-4667-8815-7efda593148c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-ddd4cb93-f581-4667-8815-7efda593148c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:09.626960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:09.627885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:09.628715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:09.628715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:09.628715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:09.628715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:09.628715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:09.628715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:09.628715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:09.628715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:09.628715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:09.628715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:09.628715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:09.628715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:09.631173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 178/340] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:09.639491 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4cb93-f581-4667-8815-7efda593148c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:09.686552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 163/341] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:37:09.686552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3f936-a4fe-4a62-aa7a-9c32e9606742 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:09.735828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:09.828124 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4cb93-f581-4667-8815-7efda593148c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:09.854734 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:09.880487 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d0e5e-1b48-4653-865b-8f4ac71670c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:09.881483 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d0e5e-1b48-4653-865b-8f4ac71670c8 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:09.881986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d0e5e-1b48-4653-865b-8f4ac71670c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:09.882517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d0e5e-1b48-4653-865b-8f4ac71670c8 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:09.884871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d0e5e-1b48-4653-865b-8f4ac71670c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:09.885617 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d0e5e-1b48-4653-865b-8f4ac71670c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:09.890368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d0e5e-1b48-4653-865b-8f4ac71670c8 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:09.907123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 179/342] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:37:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:09.910068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d0e5e-1b48-4653-865b-8f4ac71670c8 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:09.943174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3f936-a4fe-4a62-aa7a-9c32e9606742 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:10.004759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb91d72-31c7-4287-8bcc-6788c9b1cd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:10.005506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb91d72-31c7-4287-8bcc-6788c9b1cd6b None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:10.005788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb91d72-31c7-4287-8bcc-6788c9b1cd6b None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:10.006061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb91d72-31c7-4287-8bcc-6788c9b1cd6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:10.029829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb91d72-31c7-4287-8bcc-6788c9b1cd6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-bbb91d72-31c7-4287-8bcc-6788c9b1cd6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:10.042146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:10.043244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:10.044499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:10.044499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:10.044499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:10.044499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:10.044499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:10.044499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:10.044499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:10.044499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:10.044499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:10.044499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:10.044499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:10.044499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:10.044499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 164/343] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:37:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:10.050803 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb91d72-31c7-4287-8bcc-6788c9b1cd6b None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:10.142172 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d0e5e-1b48-4653-865b-8f4ac71670c8 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:10.180955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:10.209117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb91d72-31c7-4287-8bcc-6788c9b1cd6b None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:10.231912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eded91-1aa1-4a8e-83be-913be2f243c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:10.232523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eded91-1aa1-4a8e-83be-913be2f243c9 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:10.232792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eded91-1aa1-4a8e-83be-913be2f243c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:10.233056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eded91-1aa1-4a8e-83be-913be2f243c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:10.268329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe90a33-b373-4564-9be8-10bb36fd0cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:10.268917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe90a33-b373-4564-9be8-10bb36fd0cf7 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:10.269203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe90a33-b373-4564-9be8-10bb36fd0cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:10.269473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe90a33-b373-4564-9be8-10bb36fd0cf7 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:10.270142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe90a33-b373-4564-9be8-10bb36fd0cf7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:10.270442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe90a33-b373-4564-9be8-10bb36fd0cf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:10.271878 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe90a33-b373-4564-9be8-10bb36fd0cf7 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:10.280776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 165/344] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:10.281600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe90a33-b373-4564-9be8-10bb36fd0cf7 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:10.315505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:10.350752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eded91-1aa1-4a8e-83be-913be2f243c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:10.352304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eded91-1aa1-4a8e-83be-913be2f243c9 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:10.389462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe90a33-b373-4564-9be8-10bb36fd0cf7 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:10.432787 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d5ebf-5c56-4bad-94a9-f1c188a9809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:10.433606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d5ebf-5c56-4bad-94a9-f1c188a9809e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:10.434047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d5ebf-5c56-4bad-94a9-f1c188a9809e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:10.434708 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d5ebf-5c56-4bad-94a9-f1c188a9809e None admin] PATH_INFO: `/v3/roles/member` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:10.443295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:10.446867 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d5ebf-5c56-4bad-94a9-f1c188a9809e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-307d5ebf-5c56-4bad-94a9-f1c188a9809e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:10.452031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:10.453039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:10.454258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:10.454258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:10.454258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:10.454258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:10.454258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:10.454258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:10.454258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:10.454258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:10.454258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:10.454258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:10.454258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:10.454258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:10.454258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 166/345] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:10.455196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75f065-0ee2-48cc-b519-81c0e7e6710f None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:10.455786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75f065-0ee2-48cc-b519-81c0e7e6710f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:10.457561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75f065-0ee2-48cc-b519-81c0e7e6710f None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:10.457561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 167/346] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:10.458309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75f065-0ee2-48cc-b519-81c0e7e6710f None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:10.497236 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 180/347] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 12:37:10.498732 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eded91-1aa1-4a8e-83be-913be2f243c9 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:10.597655 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:10.600402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75f065-0ee2-48cc-b519-81c0e7e6710f None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:10.626762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eded91-1aa1-4a8e-83be-913be2f243c9 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:10.641216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53959a1d-876e-4208-b666-696673ce53fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:10.641850 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53959a1d-876e-4208-b666-696673ce53fd None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:10.642137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53959a1d-876e-4208-b666-696673ce53fd None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:10.642414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53959a1d-876e-4208-b666-696673ce53fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:10.665723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324f34e-46b1-4c36-a1c7-911a8d40619b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:10.666930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324f34e-46b1-4c36-a1c7-911a8d40619b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:10.667396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324f34e-46b1-4c36-a1c7-911a8d40619b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:10.667852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324f34e-46b1-4c36-a1c7-911a8d40619b None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:10.668724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324f34e-46b1-4c36-a1c7-911a8d40619b None admin] RBAC: Adding query filter params (name=member) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:10.669205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324f34e-46b1-4c36-a1c7-911a8d40619b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:10.671241 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324f34e-46b1-4c36-a1c7-911a8d40619b None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:10.683218 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 181/348] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:37:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:10.685468 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1741f1-ce77-4147-a53f-e236f1ab5450 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:10.685934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1741f1-ce77-4147-a53f-e236f1ab5450 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:10.686860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1741f1-ce77-4147-a53f-e236f1ab5450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:10.710543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-7c1741f1-ce77-4147-a53f-e236f1ab5450 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:10.713143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7c1741f1-ce77-4147-a53f-e236f1ab5450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:10.729335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:10.732554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53959a1d-876e-4208-b666-696673ce53fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:10.734096 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53959a1d-876e-4208-b666-696673ce53fd None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:10.799446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 182/349] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:10.801504 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1741f1-ce77-4147-a53f-e236f1ab5450 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:10.866529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 168/350] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:10.868765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53959a1d-876e-4208-b666-696673ce53fd None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:10.884252 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:10.979484 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1741f1-ce77-4147-a53f-e236f1ab5450 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:10.983104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:10.990800 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53959a1d-876e-4208-b666-696673ce53fd None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:11.022001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ca15b-a43b-43ce-aa4c-93808f6165d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:11.023717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ca15b-a43b-43ce-aa4c-93808f6165d1 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:11.023965 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ca15b-a43b-43ce-aa4c-93808f6165d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:11.024418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ca15b-a43b-43ce-aa4c-93808f6165d1 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:11.025575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ca15b-a43b-43ce-aa4c-93808f6165d1 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:11.027990 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ca15b-a43b-43ce-aa4c-93808f6165d1 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:11.029919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 183/351] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:10 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:11.030811 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ca15b-a43b-43ce-aa4c-93808f6165d1 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:11.044606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a6043-e144-4347-9f08-a4e845103ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:11.045233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a6043-e144-4347-9f08-a4e845103ec0 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:11.045510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a6043-e144-4347-9f08-a4e845103ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:11.046234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a6043-e144-4347-9f08-a4e845103ec0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:11.146440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a6043-e144-4347-9f08-a4e845103ec0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:11.148901 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a6043-e144-4347-9f08-a4e845103ec0 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:11.155493 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ca15b-a43b-43ce-aa4c-93808f6165d1 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:11.194703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da691a-8b64-4e06-9954-cc69b39c18e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:11.195866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da691a-8b64-4e06-9954-cc69b39c18e1 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:11.196142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da691a-8b64-4e06-9954-cc69b39c18e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:11.196410 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da691a-8b64-4e06-9954-cc69b39c18e1 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:11.197608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da691a-8b64-4e06-9954-cc69b39c18e1 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:11.199176 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da691a-8b64-4e06-9954-cc69b39c18e1 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:11.201162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 184/352] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:11 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:11.202327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d6c82-f279-4443-998d-6c2f25f419b2 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:11.202607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d6c82-f279-4443-998d-6c2f25f419b2 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:11.202880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d6c82-f279-4443-998d-6c2f25f419b2 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:11.204065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 185/353] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:11.204740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d6c82-f279-4443-998d-6c2f25f419b2 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:11.213856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:11.274016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:11.278878 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 169/354] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:11.282510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a6043-e144-4347-9f08-a4e845103ec0 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:11.351947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d6c82-f279-4443-998d-6c2f25f419b2 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:11.401535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd12189-e15a-4cc0-88f5-4d9e813563e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:11.402094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd12189-e15a-4cc0-88f5-4d9e813563e2 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:11.402744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd12189-e15a-4cc0-88f5-4d9e813563e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:11.403028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd12189-e15a-4cc0-88f5-4d9e813563e2 None admin] PATH_INFO: `/v3/groups/940b14c2156d43a9916a8e5b4311fb74` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:11.408061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a6043-e144-4347-9f08-a4e845103ec0 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:11.410619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd12189-e15a-4cc0-88f5-4d9e813563e2 None admin] RBAC: Authorizing `identity:get_group(group_id=940b14c2156d43a9916a8e5b4311fb74)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:11.411501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd12189-e15a-4cc0-88f5-4d9e813563e2 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:11.413102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 186/355] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:11 2022] GET /identity/v3/groups/940b14c2156d43a9916a8e5b4311fb74 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:11.414362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861792a-d801-4ad5-b562-3d42af3962b3 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:11.415758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861792a-d801-4ad5-b562-3d42af3962b3 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:11.416039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861792a-d801-4ad5-b562-3d42af3962b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:11.440605 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-c861792a-d801-4ad5-b562-3d42af3962b3 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:11.441330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c861792a-d801-4ad5-b562-3d42af3962b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:11.462741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465003c3-aa0b-4698-9322-00688ef2522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:11.463432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465003c3-aa0b-4698-9322-00688ef2522d None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:11.463706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465003c3-aa0b-4698-9322-00688ef2522d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:11.463978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465003c3-aa0b-4698-9322-00688ef2522d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:11.478219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465003c3-aa0b-4698-9322-00688ef2522d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-465003c3-aa0b-4698-9322-00688ef2522d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:11.483214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:11.484129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:11.484909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:11.484909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:11.484909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:11.484909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:11.484909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:11.484909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:11.484909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:11.484909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:11.484909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:11.484909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:11.484909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 12:37:11.484909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:11.484909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 170/356] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:11.485564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465003c3-aa0b-4698-9322-00688ef2522d None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:11.521043 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:11.537151 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 187/357] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:11.537151 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861792a-d801-4ad5-b562-3d42af3962b3 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:11.580140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:11.634520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465003c3-aa0b-4698-9322-00688ef2522d None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:11.670930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861792a-d801-4ad5-b562-3d42af3962b3 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:11.675110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037d609-9d46-40ec-95a2-c9c0c37f97e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:11.675805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037d609-9d46-40ec-95a2-c9c0c37f97e3 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:11.676086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037d609-9d46-40ec-95a2-c9c0c37f97e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:11.676355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037d609-9d46-40ec-95a2-c9c0c37f97e3 None admin] PATH_INFO: `/v3/groups/940b14c2156d43a9916a8e5b4311fb74` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:11.677471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037d609-9d46-40ec-95a2-c9c0c37f97e3 None admin] RBAC: Authorizing `identity:get_group(group_id=940b14c2156d43a9916a8e5b4311fb74)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:11.679469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037d609-9d46-40ec-95a2-c9c0c37f97e3 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:11.681063 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 171/358] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:11 2022] GET /identity/v3/groups/940b14c2156d43a9916a8e5b4311fb74 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:11.681882 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037d609-9d46-40ec-95a2-c9c0c37f97e3 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:11.715997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571f801-7a21-4ec2-8652-f91ab5930913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:11.717010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571f801-7a21-4ec2-8652-f91ab5930913 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:11.717292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571f801-7a21-4ec2-8652-f91ab5930913 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:11.717865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571f801-7a21-4ec2-8652-f91ab5930913 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:11.720132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571f801-7a21-4ec2-8652-f91ab5930913 None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:11.721856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571f801-7a21-4ec2-8652-f91ab5930913 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:11.724243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 188/359] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:11 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:11.725091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571f801-7a21-4ec2-8652-f91ab5930913 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:11.809656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:11.816304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037d609-9d46-40ec-95a2-c9c0c37f97e3 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:11.848199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:11.859037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a98acd-6bc9-43f7-99b3-c5452b4e73cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:11.859241 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571f801-7a21-4ec2-8652-f91ab5930913 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:11.859651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a98acd-6bc9-43f7-99b3-c5452b4e73cb None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:11.859931 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a98acd-6bc9-43f7-99b3-c5452b4e73cb None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:11.860203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a98acd-6bc9-43f7-99b3-c5452b4e73cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:11.862384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a98acd-6bc9-43f7-99b3-c5452b4e73cb None admin] RBAC: Adding query filter params (group.id=940b14c2156d43a9916a8e5b4311fb74, role.id=43667d074eda4143bf3b7f721e83810c, scope.project.id=8b0387feea344c35a5907f20e962c357) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:11.862722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a98acd-6bc9-43f7-99b3-c5452b4e73cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:11.864925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a98acd-6bc9-43f7-99b3-c5452b4e73cb None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:11.874849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 172/360] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 12:37:11 2022] GET /identity/v3/role_assignments?group.id=940b14c2156d43a9916a8e5b4311fb74&scope.project.id=8b0387feea344c35a5907f20e962c357&role.id=43667d074eda4143bf3b7f721e83810c => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:11.874849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a98acd-6bc9-43f7-99b3-c5452b4e73cb None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:11.913382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fef7f-3e02-40ca-b7e6-507e5fab23f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:11.914585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fef7f-3e02-40ca-b7e6-507e5fab23f0 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:11.915176 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fef7f-3e02-40ca-b7e6-507e5fab23f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:11.915488 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fef7f-3e02-40ca-b7e6-507e5fab23f0 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:11.916165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8fef7f-3e02-40ca-b7e6-507e5fab23f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:11.916462 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8fef7f-3e02-40ca-b7e6-507e5fab23f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:11.917894 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8fef7f-3e02-40ca-b7e6-507e5fab23f0 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:11.927302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 189/361] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:37:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:11.927895 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fef7f-3e02-40ca-b7e6-507e5fab23f0 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:12.002352 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a98acd-6bc9-43f7-99b3-c5452b4e73cb None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:12.058932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78fddc-0a22-4082-964d-d4acff885709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:12.058932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78fddc-0a22-4082-964d-d4acff885709 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:12.060030 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78fddc-0a22-4082-964d-d4acff885709 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:12.060030 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78fddc-0a22-4082-964d-d4acff885709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:12.060780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78fddc-0a22-4082-964d-d4acff885709 None admin] RBAC: Adding query filter params (group.id=940b14c2156d43a9916a8e5b4311fb74, role.id=96271a2245c144d4b43dea66f18074e8, scope.project.id=8b0387feea344c35a5907f20e962c357) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:12.061082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78fddc-0a22-4082-964d-d4acff885709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:12.065954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78fddc-0a22-4082-964d-d4acff885709 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:12.072319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 173/362] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 12:37:11 2022] GET /identity/v3/role_assignments?group.id=940b14c2156d43a9916a8e5b4311fb74&scope.project.id=8b0387feea344c35a5907f20e962c357&role.id=96271a2245c144d4b43dea66f18074e8 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:12.073364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a60fd-45ad-411f-afc1-1d2336f00d3c None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:12.073646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a60fd-45ad-411f-afc1-1d2336f00d3c None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:12.073913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a60fd-45ad-411f-afc1-1d2336f00d3c None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:12.075342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 174/363] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:12.075934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a60fd-45ad-411f-afc1-1d2336f00d3c None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:12.077381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fef7f-3e02-40ca-b7e6-507e5fab23f0 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:12.113539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:12.124631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334ff5c-5c27-461f-895e-261fe0e93d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:12.126006 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ff5c-5c27-461f-895e-261fe0e93d54 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:12.126296 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ff5c-5c27-461f-895e-261fe0e93d54 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:12.126643 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ff5c-5c27-461f-895e-261fe0e93d54 None admin] PATH_INFO: `/v3/groups/940b14c2156d43a9916a8e5b4311fb74` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:12.129582 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334ff5c-5c27-461f-895e-261fe0e93d54 None admin] RBAC: Authorizing `identity:get_group(group_id=940b14c2156d43a9916a8e5b4311fb74)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:12.131092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334ff5c-5c27-461f-895e-261fe0e93d54 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:12.133839 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 190/364] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:11 2022] GET /identity/v3/groups/940b14c2156d43a9916a8e5b4311fb74 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:12.134931 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334ff5c-5c27-461f-895e-261fe0e93d54 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:12.149860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:12.218866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a60fd-45ad-411f-afc1-1d2336f00d3c None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:12.266334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f812c3-d5d1-4d42-894e-f9e9d75bcc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:12.267280 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f812c3-d5d1-4d42-894e-f9e9d75bcc24 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:12.267890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f812c3-d5d1-4d42-894e-f9e9d75bcc24 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:12.268393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f812c3-d5d1-4d42-894e-f9e9d75bcc24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:12.282312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334ff5c-5c27-461f-895e-261fe0e93d54 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:12.286509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f812c3-d5d1-4d42-894e-f9e9d75bcc24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-a2f812c3-d5d1-4d42-894e-f9e9d75bcc24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:12.292625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:12.293581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:12.294390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:12.294390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:12.294390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:12.294390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:12.294390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:12.294390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:12.294390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:12.294390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:12.294390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:12.294390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:12.294390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:12.294390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:12.295723 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 175/365] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:37:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:12.296908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf8a95-3796-47b5-ad3f-c2912ceb40e7 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:12.297216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf8a95-3796-47b5-ad3f-c2912ceb40e7 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:12.297487 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf8a95-3796-47b5-ad3f-c2912ceb40e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:12.321363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-cddf8a95-3796-47b5-ad3f-c2912ceb40e7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:12.323463 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-cddf8a95-3796-47b5-ad3f-c2912ceb40e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:12.332781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139fccd-c75a-4eed-9251-15d783afeaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:12.333726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139fccd-c75a-4eed-9251-15d783afeaa3 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:12.333726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139fccd-c75a-4eed-9251-15d783afeaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:12.333964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139fccd-c75a-4eed-9251-15d783afeaa3 None admin] PATH_INFO: `/v3/users/3f96c267d5ae4563847bcbe8d26ddfad` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:12.335556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139fccd-c75a-4eed-9251-15d783afeaa3 None admin] RBAC: Authorizing `identity:get_user(user_id=3f96c267d5ae4563847bcbe8d26ddfad)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:12.338038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139fccd-c75a-4eed-9251-15d783afeaa3 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:12.341722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 191/366] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:37:12 2022] GET /identity/v3/users/3f96c267d5ae4563847bcbe8d26ddfad => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:12.342524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139fccd-c75a-4eed-9251-15d783afeaa3 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:12.414857 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 176/367] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:12.415956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf8a95-3796-47b5-ad3f-c2912ceb40e7 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:12.428536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:12.475159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139fccd-c75a-4eed-9251-15d783afeaa3 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:12.478300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:12.542021 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a84bf-fbf7-45bf-a780-3748f8805779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:12.542742 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a84bf-fbf7-45bf-a780-3748f8805779 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:12.543019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a84bf-fbf7-45bf-a780-3748f8805779 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:12.543288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a84bf-fbf7-45bf-a780-3748f8805779 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:12.544012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a84bf-fbf7-45bf-a780-3748f8805779 None admin] RBAC: Adding query filter params (group.id=940b14c2156d43a9916a8e5b4311fb74, role.id=43667d074eda4143bf3b7f721e83810c, scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:12.544314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a84bf-fbf7-45bf-a780-3748f8805779 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:12.545872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a84bf-fbf7-45bf-a780-3748f8805779 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:12.550462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf8a95-3796-47b5-ad3f-c2912ceb40e7 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:12.556207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 192/368] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 12:37:12 2022] GET /identity/v3/role_assignments?group.id=940b14c2156d43a9916a8e5b4311fb74&scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30&role.id=43667d074eda4143bf3b7f721e83810c => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:12.557048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a84bf-fbf7-45bf-a780-3748f8805779 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:12.591048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356a489-a046-426f-a0e7-efb7866ebd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:12.591670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356a489-a046-426f-a0e7-efb7866ebd6f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:12.591947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356a489-a046-426f-a0e7-efb7866ebd6f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:12.592219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356a489-a046-426f-a0e7-efb7866ebd6f None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:12.592879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356a489-a046-426f-a0e7-efb7866ebd6f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:12.593174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356a489-a046-426f-a0e7-efb7866ebd6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:12.594592 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356a489-a046-426f-a0e7-efb7866ebd6f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:12.604183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 177/369] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:12.610379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356a489-a046-426f-a0e7-efb7866ebd6f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:12.745080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a84bf-fbf7-45bf-a780-3748f8805779 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:12.787751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356a489-a046-426f-a0e7-efb7866ebd6f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:12.798841 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:12.804857 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:12.813641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f7e36-52d6-4dca-8b76-d666f8e51f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:12.814266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f7e36-52d6-4dca-8b76-d666f8e51f77 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:12.814567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f7e36-52d6-4dca-8b76-d666f8e51f77 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:12.815732 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f7e36-52d6-4dca-8b76-d666f8e51f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:12.821665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f7e36-52d6-4dca-8b76-d666f8e51f77 None admin] RBAC: Adding query filter params (role.id=ca39d64092424a34aaaf4818ca16e988, user.id=3f96c267d5ae4563847bcbe8d26ddfad, scope.system=all) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:12.821665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f7e36-52d6-4dca-8b76-d666f8e51f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:12.821665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f7e36-52d6-4dca-8b76-d666f8e51f77 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:12.842727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 193/370] 10.222.0.46 () {56 vars in 1308 bytes} [Fri Sep 9 12:37:12 2022] GET /identity/v3/role_assignments?user.id=3f96c267d5ae4563847bcbe8d26ddfad&scope.system=all&role.id=ca39d64092424a34aaaf4818ca16e988 => generated 510 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:12.853428 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e075fd-1cdd-48a6-a086-ab25955654f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:12.855138 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e075fd-1cdd-48a6-a086-ab25955654f1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:12.855138 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e075fd-1cdd-48a6-a086-ab25955654f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:12.855138 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e075fd-1cdd-48a6-a086-ab25955654f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:12.945002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e075fd-1cdd-48a6-a086-ab25955654f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:12.947196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e075fd-1cdd-48a6-a086-ab25955654f1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:12.949001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f7e36-52d6-4dca-8b76-d666f8e51f77 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:13.151396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 178/371] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:13.156966 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e075fd-1cdd-48a6-a086-ab25955654f1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:13.158355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f7e36-52d6-4dca-8b76-d666f8e51f77 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:13.170876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:13.197701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc1870-3868-4b90-8fc1-25b5fa448be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:13.198339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc1870-3868-4b90-8fc1-25b5fa448be1 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:13.198641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc1870-3868-4b90-8fc1-25b5fa448be1 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:13.198970 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc1870-3868-4b90-8fc1-25b5fa448be1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:13.209698 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc1870-3868-4b90-8fc1-25b5fa448be1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-3afc1870-3868-4b90-8fc1-25b5fa448be1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:13.215826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:13.216728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:13.218056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:13.218056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:13.218056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:13.218056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:13.218056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:13.218056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:13.218056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:13.218056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:13.218056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:13.218056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:13.218056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:37:13.218056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:13.218056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 194/372] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:37:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:37:13.225372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc1870-3868-4b90-8fc1-25b5fa448be1 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:13.296697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:13.300444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e075fd-1cdd-48a6-a086-ab25955654f1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:13.343294 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f423a-b995-461d-838b-8ce3a816b820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:13.344514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f423a-b995-461d-838b-8ce3a816b820 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:13.344930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f423a-b995-461d-838b-8ce3a816b820 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:13.345334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f423a-b995-461d-838b-8ce3a816b820 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:13.346960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f423a-b995-461d-838b-8ce3a816b820 None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:13.348958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f423a-b995-461d-838b-8ce3a816b820 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:13.351064 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 179/373] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:13 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:13.356937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f423a-b995-461d-838b-8ce3a816b820 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:13.384095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc1870-3868-4b90-8fc1-25b5fa448be1 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:13.450547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248b427-775b-420e-8df4-31a5cdd9553f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:13.451458 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248b427-775b-420e-8df4-31a5cdd9553f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:13.451886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248b427-775b-420e-8df4-31a5cdd9553f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:13.452372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248b427-775b-420e-8df4-31a5cdd9553f None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:13.453362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248b427-775b-420e-8df4-31a5cdd9553f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:13.453919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248b427-775b-420e-8df4-31a5cdd9553f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:13.455487 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248b427-775b-420e-8df4-31a5cdd9553f None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:13.474411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:13.475426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 195/374] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:13.486102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248b427-775b-420e-8df4-31a5cdd9553f None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:13.506332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f423a-b995-461d-838b-8ce3a816b820 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:13.565073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556623b-ae13-42af-966e-10fe163ffb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:13.565771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556623b-ae13-42af-966e-10fe163ffb89 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:13.566065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556623b-ae13-42af-966e-10fe163ffb89 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:13.566351 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556623b-ae13-42af-966e-10fe163ffb89 None admin] PATH_INFO: `/v3/groups/940b14c2156d43a9916a8e5b4311fb74` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:13.567501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556623b-ae13-42af-966e-10fe163ffb89 None admin] RBAC: Authorizing `identity:get_group(group_id=940b14c2156d43a9916a8e5b4311fb74)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:13.570997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556623b-ae13-42af-966e-10fe163ffb89 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:13.572607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 180/375] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:13 2022] GET /identity/v3/groups/940b14c2156d43a9916a8e5b4311fb74 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:13.581579 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556623b-ae13-42af-966e-10fe163ffb89 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:13.690369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248b427-775b-420e-8df4-31a5cdd9553f None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:13.703059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:13.764539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addee244-b6ef-47ae-add8-cabc680d9607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:13.765403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addee244-b6ef-47ae-add8-cabc680d9607 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:13.765696 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addee244-b6ef-47ae-add8-cabc680d9607 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:13.765972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addee244-b6ef-47ae-add8-cabc680d9607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:13.768193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556623b-ae13-42af-966e-10fe163ffb89 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:13.840473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febfc27-706c-4fd8-bd14-de1f3746c080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:13.841097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febfc27-706c-4fd8-bd14-de1f3746c080 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:13.841377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febfc27-706c-4fd8-bd14-de1f3746c080 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:13.841695 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febfc27-706c-4fd8-bd14-de1f3746c080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:13.842433 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febfc27-706c-4fd8-bd14-de1f3746c080 None admin] RBAC: Adding query filter params (group.id=940b14c2156d43a9916a8e5b4311fb74, role.id=96271a2245c144d4b43dea66f18074e8, scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:13.842770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febfc27-706c-4fd8-bd14-de1f3746c080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:13.845605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febfc27-706c-4fd8-bd14-de1f3746c080 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:13.856198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 181/376] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 12:37:13 2022] GET /identity/v3/role_assignments?group.id=940b14c2156d43a9916a8e5b4311fb74&scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30&role.id=96271a2245c144d4b43dea66f18074e8 => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:13.890469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addee244-b6ef-47ae-add8-cabc680d9607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:13.892033 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addee244-b6ef-47ae-add8-cabc680d9607 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:14.034169 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 196/377] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 12:37:14.047729 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febfc27-706c-4fd8-bd14-de1f3746c080 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:14.065277 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:14.194187 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febfc27-706c-4fd8-bd14-de1f3746c080 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:14.237628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46722d53-8436-4a8b-a638-001b22f60a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:14.238268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46722d53-8436-4a8b-a638-001b22f60a86 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:14.238589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46722d53-8436-4a8b-a638-001b22f60a86 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:14.238864 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46722d53-8436-4a8b-a638-001b22f60a86 None admin] PATH_INFO: `/v3/projects/e314473e6b1442dfa9c0bcc6fc5f97c6` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:14.239985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46722d53-8436-4a8b-a638-001b22f60a86 None admin] RBAC: Authorizing `identity:get_project(project_id=e314473e6b1442dfa9c0bcc6fc5f97c6)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:14.241422 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46722d53-8436-4a8b-a638-001b22f60a86 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:14.244286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 182/378] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:14 2022] GET /identity/v3/projects/e314473e6b1442dfa9c0bcc6fc5f97c6 => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:14.249853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addee244-b6ef-47ae-add8-cabc680d9607 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:14.262837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:14.406837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addee244-b6ef-47ae-add8-cabc680d9607 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:14.448173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd350d8-5d73-43d3-9124-69d036b3442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:14.448871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd350d8-5d73-43d3-9124-69d036b3442e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:14.449083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd350d8-5d73-43d3-9124-69d036b3442e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:14.449354 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd350d8-5d73-43d3-9124-69d036b3442e None admin] PATH_INFO: `/v3/groups/2a0fbe291c304fe0bd92d0dec4ec1ecc` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:14.454504 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd350d8-5d73-43d3-9124-69d036b3442e None admin] RBAC: Authorizing `identity:get_group(group_id=2a0fbe291c304fe0bd92d0dec4ec1ecc)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:14.455925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd350d8-5d73-43d3-9124-69d036b3442e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:14.457568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 197/379] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:14 2022] GET /identity/v3/groups/2a0fbe291c304fe0bd92d0dec4ec1ecc => generated 229 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:14.464362 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46722d53-8436-4a8b-a638-001b22f60a86 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:14.559708 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:14.587843 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46722d53-8436-4a8b-a638-001b22f60a86 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:14.633021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb093b-ed9f-474c-9e11-6379bd6f78bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:14.633632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb093b-ed9f-474c-9e11-6379bd6f78bc None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:14.633912 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb093b-ed9f-474c-9e11-6379bd6f78bc None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:14.634184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb093b-ed9f-474c-9e11-6379bd6f78bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:14.634989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb093b-ed9f-474c-9e11-6379bd6f78bc None admin] RBAC: Adding query filter params (group.id=2a0fbe291c304fe0bd92d0dec4ec1ecc, role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=e314473e6b1442dfa9c0bcc6fc5f97c6) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:14.635351 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb093b-ed9f-474c-9e11-6379bd6f78bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:14.636789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb093b-ed9f-474c-9e11-6379bd6f78bc None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:14.642764 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 183/380] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 12:37:14 2022] GET /identity/v3/role_assignments?group.id=2a0fbe291c304fe0bd92d0dec4ec1ecc&scope.project.id=e314473e6b1442dfa9c0bcc6fc5f97c6&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:15.028783 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6219-9907-409b-9ff3-478834cc168a None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:15.028783 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6219-9907-409b-9ff3-478834cc168a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:15.028783 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6219-9907-409b-9ff3-478834cc168a None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:15.029958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 198/381] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:15.037625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d4fee-114e-4577-a54a-996ffc62b16d None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:15.037625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d4fee-114e-4577-a54a-996ffc62b16d None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:15.037625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d4fee-114e-4577-a54a-996ffc62b16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:15.063525 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-9d7d4fee-114e-4577-a54a-996ffc62b16d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:15.065096 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9d7d4fee-114e-4577-a54a-996ffc62b16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:15.105049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0304c-26eb-48dd-a990-ed06705b3963 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:15.105367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0304c-26eb-48dd-a990-ed06705b3963 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:15.105590 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0304c-26eb-48dd-a990-ed06705b3963 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:15.110538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 199/382] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:15.111976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f7dc6-eae2-46ee-ad40-eb31e9b1a3bd None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:15.112251 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f7dc6-eae2-46ee-ad40-eb31e9b1a3bd None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:15.112581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f7dc6-eae2-46ee-ad40-eb31e9b1a3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:15.149537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-f49f7dc6-eae2-46ee-ad40-eb31e9b1a3bd None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:15.153048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f49f7dc6-eae2-46ee-ad40-eb31e9b1a3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:15.202427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 184/383] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:15.230958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:15.238803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 200/384] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:37:15.623817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d4fee-114e-4577-a54a-996ffc62b16d None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:15.650049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f7dc6-eae2-46ee-ad40-eb31e9b1a3bd None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:15.712372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:15.742981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d4fee-114e-4577-a54a-996ffc62b16d None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:15.789453 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ccead-d0f9-46d1-b671-fc0252f6b50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:15.790406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ccead-d0f9-46d1-b671-fc0252f6b50a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:15.790756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ccead-d0f9-46d1-b671-fc0252f6b50a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:15.791054 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ccead-d0f9-46d1-b671-fc0252f6b50a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:15.803658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ccead-d0f9-46d1-b671-fc0252f6b50a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-4b5ccead-d0f9-46d1-b671-fc0252f6b50a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:15.809827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:15.810790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:15.811623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:15.811623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:15.811623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:15.811623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:15.811623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:15.811623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:15.811623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:15.811623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:15.811623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:15.811623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:15.811623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:15.811623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:15.812269 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 185/385] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:37:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:15.818050 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ccead-d0f9-46d1-b671-fc0252f6b50a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:15.871278 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f7dc6-eae2-46ee-ad40-eb31e9b1a3bd None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:15.909139 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f3784-830d-4504-8e5e-6b13b92da7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:15.909875 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f3784-830d-4504-8e5e-6b13b92da7c0 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:15.910162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f3784-830d-4504-8e5e-6b13b92da7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:15.910434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f3784-830d-4504-8e5e-6b13b92da7c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:15.922639 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f3784-830d-4504-8e5e-6b13b92da7c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-194f3784-830d-4504-8e5e-6b13b92da7c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:15.927499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:15.928412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:15.929204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:15.929204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:15.929204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:15.929204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:15.929204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:15.929204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:15.929204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:15.929204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:15.929204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:15.929204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:15.929204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:15.929204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:15.929204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 201/386] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:37:15.936992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f3784-830d-4504-8e5e-6b13b92da7c0 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:16.001388 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ccead-d0f9-46d1-b671-fc0252f6b50a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:16.007388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:16.054385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f3784-830d-4504-8e5e-6b13b92da7c0 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:16.067095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:16.077522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee0682-0344-4e22-9c2b-350fea5c6084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:16.078193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee0682-0344-4e22-9c2b-350fea5c6084 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:16.078474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee0682-0344-4e22-9c2b-350fea5c6084 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:16.078766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee0682-0344-4e22-9c2b-350fea5c6084 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:16.079428 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee0682-0344-4e22-9c2b-350fea5c6084 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:16.079726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee0682-0344-4e22-9c2b-350fea5c6084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:16.081092 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee0682-0344-4e22-9c2b-350fea5c6084 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:16.094163 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc75afb-08a9-4e67-a306-d5a573c3c996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:16.094658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 186/387] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:16.094830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc75afb-08a9-4e67-a306-d5a573c3c996 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:16.095087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc75afb-08a9-4e67-a306-d5a573c3c996 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:16.095365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc75afb-08a9-4e67-a306-d5a573c3c996 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:16.096050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc75afb-08a9-4e67-a306-d5a573c3c996 None admin] RBAC: Adding query filter params (name=member) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:16.096369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc75afb-08a9-4e67-a306-d5a573c3c996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:16.098088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc75afb-08a9-4e67-a306-d5a573c3c996 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:16.112926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 202/388] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:37:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:16.113231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee0682-0344-4e22-9c2b-350fea5c6084 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:16.113600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc75afb-08a9-4e67-a306-d5a573c3c996 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:16.236303 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc75afb-08a9-4e67-a306-d5a573c3c996 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:16.266022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:16.278280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9ef93-1685-4753-ba7d-d16396a2ebfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:16.278949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9ef93-1685-4753-ba7d-d16396a2ebfc None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:16.279255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9ef93-1685-4753-ba7d-d16396a2ebfc None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:16.279505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9ef93-1685-4753-ba7d-d16396a2ebfc None admin] PATH_INFO: `/v3/groups/940b14c2156d43a9916a8e5b4311fb74` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:16.280634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9ef93-1685-4753-ba7d-d16396a2ebfc None admin] RBAC: Authorizing `identity:get_group(group_id=940b14c2156d43a9916a8e5b4311fb74)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:16.280954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee0682-0344-4e22-9c2b-350fea5c6084 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:16.282070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9ef93-1685-4753-ba7d-d16396a2ebfc None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:16.289672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 203/389] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:16 2022] GET /identity/v3/groups/940b14c2156d43a9916a8e5b4311fb74 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:16.290651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9ef93-1685-4753-ba7d-d16396a2ebfc None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:16.371210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773faf85-23de-4fdd-9797-02beec568afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:16.371862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773faf85-23de-4fdd-9797-02beec568afa None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:16.372147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773faf85-23de-4fdd-9797-02beec568afa None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:16.372421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773faf85-23de-4fdd-9797-02beec568afa None admin] PATH_INFO: `/v3/groups/940b14c2156d43a9916a8e5b4311fb74` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:16.374688 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773faf85-23de-4fdd-9797-02beec568afa None admin] RBAC: Authorizing `identity:get_group(group_id=940b14c2156d43a9916a8e5b4311fb74)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:16.376114 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773faf85-23de-4fdd-9797-02beec568afa None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:16.377856 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 187/390] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:16 2022] GET /identity/v3/groups/940b14c2156d43a9916a8e5b4311fb74 => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:16.384434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773faf85-23de-4fdd-9797-02beec568afa None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:16.408277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9ef93-1685-4753-ba7d-d16396a2ebfc None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:16.451230 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056f681-7926-4af0-9e11-db926809d862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:16.451877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056f681-7926-4af0-9e11-db926809d862 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:16.452154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056f681-7926-4af0-9e11-db926809d862 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:16.452420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056f681-7926-4af0-9e11-db926809d862 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:16.454572 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056f681-7926-4af0-9e11-db926809d862 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:16.456000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056f681-7926-4af0-9e11-db926809d862 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:16.457915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 204/391] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:16 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:16.459084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867d982-cfd5-48c9-b6cc-92b851d665a7 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:16.459363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867d982-cfd5-48c9-b6cc-92b851d665a7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:16.459623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867d982-cfd5-48c9-b6cc-92b851d665a7 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:16.461169 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 205/392] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:16.467311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df0e7e-aab2-46d9-a7e4-7513c5aa81b5 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:16.467596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df0e7e-aab2-46d9-a7e4-7513c5aa81b5 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:16.467872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df0e7e-aab2-46d9-a7e4-7513c5aa81b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:16.488077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:16.491182 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-44df0e7e-aab2-46d9-a7e4-7513c5aa81b5 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:16.493004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-44df0e7e-aab2-46d9-a7e4-7513c5aa81b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:16.540273 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773faf85-23de-4fdd-9797-02beec568afa None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:16.561785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:16.578726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 206/393] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:16.579607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df0e7e-aab2-46d9-a7e4-7513c5aa81b5 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:16.592806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cea406-072c-418f-bb96-b5ba05abb0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:16.593506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cea406-072c-418f-bb96-b5ba05abb0d1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:16.594616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cea406-072c-418f-bb96-b5ba05abb0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:16.594892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cea406-072c-418f-bb96-b5ba05abb0d1 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:16.599514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cea406-072c-418f-bb96-b5ba05abb0d1 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:16.601105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cea406-072c-418f-bb96-b5ba05abb0d1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:16.603755 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 188/394] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:16 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:16.611077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cea406-072c-418f-bb96-b5ba05abb0d1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:16.701693 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df0e7e-aab2-46d9-a7e4-7513c5aa81b5 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:16.749886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531474ee-59f6-49d9-a76a-68baad469f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:16.750508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531474ee-59f6-49d9-a76a-68baad469f4d None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:16.750796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531474ee-59f6-49d9-a76a-68baad469f4d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:16.751068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531474ee-59f6-49d9-a76a-68baad469f4d None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357/groups/940b14c2156d43a9916a8e5b4311fb74/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:16.753570 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531474ee-59f6-49d9-a76a-68baad469f4d None admin] RBAC: Authorizing `identity:create_grant(project_id=8b0387feea344c35a5907f20e962c357, group_id=940b14c2156d43a9916a8e5b4311fb74, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:16.755517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531474ee-59f6-49d9-a76a-68baad469f4d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:16.779928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cea406-072c-418f-bb96-b5ba05abb0d1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:16.807981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:16.815940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfaf155-26a6-4223-b32a-3970bc2b5e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:16.816601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaf155-26a6-4223-b32a-3970bc2b5e40 None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:16.817067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaf155-26a6-4223-b32a-3970bc2b5e40 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:16.817337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaf155-26a6-4223-b32a-3970bc2b5e40 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357/groups/940b14c2156d43a9916a8e5b4311fb74/roles/96271a2245c144d4b43dea66f18074e8` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:16.819520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfaf155-26a6-4223-b32a-3970bc2b5e40 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b0387feea344c35a5907f20e962c357, group_id=940b14c2156d43a9916a8e5b4311fb74, role_id=96271a2245c144d4b43dea66f18074e8)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:16.821042 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfaf155-26a6-4223-b32a-3970bc2b5e40 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:16.846604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 207/395] 10.222.0.46 () {58 vars in 1442 bytes} [Fri Sep 9 12:37:16 2022] PUT /identity/v3/projects/8b0387feea344c35a5907f20e962c357/groups/940b14c2156d43a9916a8e5b4311fb74/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:37:16.847898 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100cfc0-b6f4-445f-8b3d-f33a5dfea441 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:16.848189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100cfc0-b6f4-445f-8b3d-f33a5dfea441 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:16.848464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100cfc0-b6f4-445f-8b3d-f33a5dfea441 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:16.849862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 208/396] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:16.856574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2f304-2e79-40de-88d5-79ee1b1e8916 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:16.856865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2f304-2e79-40de-88d5-79ee1b1e8916 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:16.858512 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2f304-2e79-40de-88d5-79ee1b1e8916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:16.878242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d4a2f304-2e79-40de-88d5-79ee1b1e8916 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:16.879997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d4a2f304-2e79-40de-88d5-79ee1b1e8916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:16.908820 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 189/397] 10.222.0.46 () {58 vars in 1442 bytes} [Fri Sep 9 12:37:16 2022] PUT /identity/v3/projects/8b0387feea344c35a5907f20e962c357/groups/940b14c2156d43a9916a8e5b4311fb74/roles/96271a2245c144d4b43dea66f18074e8 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:37:16.925713 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfaf155-26a6-4223-b32a-3970bc2b5e40 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:16.977066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:17.037400 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 209/398] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:17.090424 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfaf155-26a6-4223-b32a-3970bc2b5e40 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:17.132079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f372e-8577-4db0-b785-41962c217078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:17.133340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f372e-8577-4db0-b785-41962c217078 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:17.133628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f372e-8577-4db0-b785-41962c217078 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:17.133900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f372e-8577-4db0-b785-41962c217078 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:17.146622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f372e-8577-4db0-b785-41962c217078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-ce5f372e-8577-4db0-b785-41962c217078 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:17.151586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:17.152655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:17.153671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:17.153671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:17.153671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:17.153671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:17.153671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:17.153671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:17.153671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:17.153671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:17.153671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:17.153671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:17.153671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:17.153671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:17.153671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 190/399] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:37:17.158978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2f304-2e79-40de-88d5-79ee1b1e8916 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:17.273862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2f304-2e79-40de-88d5-79ee1b1e8916 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:17.311490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c353e3f-f25f-4509-b465-2daca494f722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:17.312025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c353e3f-f25f-4509-b465-2daca494f722 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:17.312297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c353e3f-f25f-4509-b465-2daca494f722 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:17.312565 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c353e3f-f25f-4509-b465-2daca494f722 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:17.313237 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c353e3f-f25f-4509-b465-2daca494f722 None admin] RBAC: Adding query filter params (name=member) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:17.313529 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c353e3f-f25f-4509-b465-2daca494f722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:17.314917 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c353e3f-f25f-4509-b465-2daca494f722 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:17.324089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 210/400] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:37:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:17.329540 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f372e-8577-4db0-b785-41962c217078 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:17.350591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105aa69b-a411-4fef-beb4-6011b52f9ac7 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:17.350876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105aa69b-a411-4fef-beb4-6011b52f9ac7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:17.351146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105aa69b-a411-4fef-beb4-6011b52f9ac7 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:17.353341 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 211/401] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:37:17.358635 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f900bfd-6705-4052-8978-b8be9a72797b None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:17.359089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f900bfd-6705-4052-8978-b8be9a72797b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:17.359500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f900bfd-6705-4052-8978-b8be9a72797b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:17.383138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-1f900bfd-6705-4052-8978-b8be9a72797b None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:17.385375 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1f900bfd-6705-4052-8978-b8be9a72797b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:17.404072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:17.425909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:17.455713 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f372e-8577-4db0-b785-41962c217078 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:17.473692 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 212/402] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:37:17.473692 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f900bfd-6705-4052-8978-b8be9a72797b None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:17.510583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bafa8-fed4-42b7-8cb9-01ae2afc15cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:17.511639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bafa8-fed4-42b7-8cb9-01ae2afc15cd None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:17.512158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bafa8-fed4-42b7-8cb9-01ae2afc15cd None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:17.512625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bafa8-fed4-42b7-8cb9-01ae2afc15cd None admin] PATH_INFO: `/v3/groups/940b14c2156d43a9916a8e5b4311fb74` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:17.513953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bafa8-fed4-42b7-8cb9-01ae2afc15cd None admin] RBAC: Authorizing `identity:get_group(group_id=940b14c2156d43a9916a8e5b4311fb74)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:17.522786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bafa8-fed4-42b7-8cb9-01ae2afc15cd None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:17.527015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 191/403] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:17 2022] GET /identity/v3/groups/940b14c2156d43a9916a8e5b4311fb74 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:17.531861 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bafa8-fed4-42b7-8cb9-01ae2afc15cd None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:17.607155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f900bfd-6705-4052-8978-b8be9a72797b None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:17.647565 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbaf586-9832-486c-ae9e-58ffe928ec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:17.648822 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbaf586-9832-486c-ae9e-58ffe928ec84 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:17.649215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbaf586-9832-486c-ae9e-58ffe928ec84 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:17.649492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbaf586-9832-486c-ae9e-58ffe928ec84 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:17.679385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbaf586-9832-486c-ae9e-58ffe928ec84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-cbbaf586-9832-486c-ae9e-58ffe928ec84 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:17.689205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:17.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 213/404] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:37:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:17.699374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbaf586-9832-486c-ae9e-58ffe928ec84 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:17.750828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:17.751057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bafa8-fed4-42b7-8cb9-01ae2afc15cd None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:17.781538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:17.807222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28541197-43f7-41ab-aced-cea2ad5824fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:17.807858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28541197-43f7-41ab-aced-cea2ad5824fc None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:17.808253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28541197-43f7-41ab-aced-cea2ad5824fc None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:17.808527 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28541197-43f7-41ab-aced-cea2ad5824fc None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:17.810177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28541197-43f7-41ab-aced-cea2ad5824fc None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:17.811666 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28541197-43f7-41ab-aced-cea2ad5824fc None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:17.813524 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 192/405] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:17 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:17.814427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28541197-43f7-41ab-aced-cea2ad5824fc None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:17.850907 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbaf586-9832-486c-ae9e-58ffe928ec84 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:17.896577 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2bba3-e728-4a35-b3a0-fccd1d1939e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:17.897370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2bba3-e728-4a35-b3a0-fccd1d1939e4 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:17.897648 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2bba3-e728-4a35-b3a0-fccd1d1939e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:17.897921 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2bba3-e728-4a35-b3a0-fccd1d1939e4 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:17.898652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2bba3-e728-4a35-b3a0-fccd1d1939e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:17.898956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2bba3-e728-4a35-b3a0-fccd1d1939e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:17.900579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2bba3-e728-4a35-b3a0-fccd1d1939e4 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:17.913403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 214/406] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:17.915445 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2bba3-e728-4a35-b3a0-fccd1d1939e4 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:17.945308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28541197-43f7-41ab-aced-cea2ad5824fc None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:17.980160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75154e23-d235-4e98-b870-5bf87ffa644f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:17.980800 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75154e23-d235-4e98-b870-5bf87ffa644f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:17.981079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75154e23-d235-4e98-b870-5bf87ffa644f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:17.981348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75154e23-d235-4e98-b870-5bf87ffa644f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:17.992573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75154e23-d235-4e98-b870-5bf87ffa644f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-75154e23-d235-4e98-b870-5bf87ffa644f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:17.998234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:17.999228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:18.000023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:18.000023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:18.000023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:18.000023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:18.000023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:18.000023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:18.000023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:18.000023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:18.000023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:18.000023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:18.000023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:37:18.000023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:18.000023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 193/407] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:37:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:18.000717 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75154e23-d235-4e98-b870-5bf87ffa644f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:18.026053 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:18.036369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2bba3-e728-4a35-b3a0-fccd1d1939e4 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:18.051121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:18.077446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dcc37-5c2c-489e-a64d-8b204df44914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:18.078396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dcc37-5c2c-489e-a64d-8b204df44914 None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:18.078854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dcc37-5c2c-489e-a64d-8b204df44914 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:18.079266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dcc37-5c2c-489e-a64d-8b204df44914 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30/groups/940b14c2156d43a9916a8e5b4311fb74/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:18.081380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dcc37-5c2c-489e-a64d-8b204df44914 None admin] RBAC: Authorizing `identity:create_grant(project_id=3cab176a56624fcbb4fd9cf5ad72ab30, group_id=940b14c2156d43a9916a8e5b4311fb74, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:18.082979 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dcc37-5c2c-489e-a64d-8b204df44914 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:18.118915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75154e23-d235-4e98-b870-5bf87ffa644f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:18.157265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cab4a-79cf-482c-92b1-409d9b35f862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:18.157811 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cab4a-79cf-482c-92b1-409d9b35f862 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:18.158086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cab4a-79cf-482c-92b1-409d9b35f862 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:18.158415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cab4a-79cf-482c-92b1-409d9b35f862 None admin] PATH_INFO: `/v3/groups/940b14c2156d43a9916a8e5b4311fb74` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:18.159550 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cab4a-79cf-482c-92b1-409d9b35f862 None admin] RBAC: Authorizing `identity:get_group(group_id=940b14c2156d43a9916a8e5b4311fb74)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:18.160961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cab4a-79cf-482c-92b1-409d9b35f862 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:18.162711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 194/408] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:17 2022] GET /identity/v3/groups/940b14c2156d43a9916a8e5b4311fb74 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:18.168449 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cab4a-79cf-482c-92b1-409d9b35f862 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:18.200356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 215/409] 10.222.0.46 () {58 vars in 1442 bytes} [Fri Sep 9 12:37:17 2022] PUT /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30/groups/940b14c2156d43a9916a8e5b4311fb74/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:37:18.207520 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dcc37-5c2c-489e-a64d-8b204df44914 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:18.337571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dcc37-5c2c-489e-a64d-8b204df44914 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:18.386526 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fb5e6-59f6-4a85-a7c4-846c1622aec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:18.387096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fb5e6-59f6-4a85-a7c4-846c1622aec7 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:18.387372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fb5e6-59f6-4a85-a7c4-846c1622aec7 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:18.387641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fb5e6-59f6-4a85-a7c4-846c1622aec7 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:18.389527 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fb5e6-59f6-4a85-a7c4-846c1622aec7 None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:18.392216 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fb5e6-59f6-4a85-a7c4-846c1622aec7 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:18.394912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 216/410] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:18 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:18.401196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cab4a-79cf-482c-92b1-409d9b35f862 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:18.402604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fb5e6-59f6-4a85-a7c4-846c1622aec7 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:18.416281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:18.427793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:18.440004 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff933c-1fe6-4c2a-9b90-412e60a50339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:18.440855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff933c-1fe6-4c2a-9b90-412e60a50339 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:18.441357 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff933c-1fe6-4c2a-9b90-412e60a50339 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:18.441780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff933c-1fe6-4c2a-9b90-412e60a50339 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:18.442642 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff933c-1fe6-4c2a-9b90-412e60a50339 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:18.443118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff933c-1fe6-4c2a-9b90-412e60a50339 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:18.444753 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff933c-1fe6-4c2a-9b90-412e60a50339 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:18.455065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 195/411] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:18.461215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff933c-1fe6-4c2a-9b90-412e60a50339 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:18.534428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fb5e6-59f6-4a85-a7c4-846c1622aec7 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:18.574401 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff933c-1fe6-4c2a-9b90-412e60a50339 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:18.581131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad7ad0-2050-43bd-9160-b0aeef39b49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:18.582030 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad7ad0-2050-43bd-9160-b0aeef39b49c None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:18.582843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad7ad0-2050-43bd-9160-b0aeef39b49c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:18.583118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad7ad0-2050-43bd-9160-b0aeef39b49c None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30/groups/940b14c2156d43a9916a8e5b4311fb74/roles/96271a2245c144d4b43dea66f18074e8` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:18.589174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad7ad0-2050-43bd-9160-b0aeef39b49c None admin] RBAC: Authorizing `identity:create_grant(project_id=3cab176a56624fcbb4fd9cf5ad72ab30, group_id=940b14c2156d43a9916a8e5b4311fb74, role_id=96271a2245c144d4b43dea66f18074e8)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:18.590629 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad7ad0-2050-43bd-9160-b0aeef39b49c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:18.614403 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38f99f-c089-4c07-b883-902bb0723be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:18.615350 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38f99f-c089-4c07-b883-902bb0723be1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:18.615864 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38f99f-c089-4c07-b883-902bb0723be1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:18.616309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38f99f-c089-4c07-b883-902bb0723be1 None admin] PATH_INFO: `/v3/groups/2a0fbe291c304fe0bd92d0dec4ec1ecc` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:18.618216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38f99f-c089-4c07-b883-902bb0723be1 None admin] RBAC: Authorizing `identity:get_group(group_id=2a0fbe291c304fe0bd92d0dec4ec1ecc)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:18.619842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38f99f-c089-4c07-b883-902bb0723be1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:18.621648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 196/412] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:18 2022] GET /identity/v3/groups/2a0fbe291c304fe0bd92d0dec4ec1ecc => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:18.627598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38f99f-c089-4c07-b883-902bb0723be1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:18.671932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:18.688746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 217/413] 10.222.0.46 () {58 vars in 1442 bytes} [Fri Sep 9 12:37:18 2022] PUT /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30/groups/940b14c2156d43a9916a8e5b4311fb74/roles/96271a2245c144d4b43dea66f18074e8 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:37:18.752646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38f99f-c089-4c07-b883-902bb0723be1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:18.796731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e345719-24b1-4fb2-833a-131165b49496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:18.797378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e345719-24b1-4fb2-833a-131165b49496 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:18.797654 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e345719-24b1-4fb2-833a-131165b49496 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:18.797926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e345719-24b1-4fb2-833a-131165b49496 None admin] PATH_INFO: `/v3/projects/e314473e6b1442dfa9c0bcc6fc5f97c6` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:18.799064 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e345719-24b1-4fb2-833a-131165b49496 None admin] RBAC: Authorizing `identity:get_project(project_id=e314473e6b1442dfa9c0bcc6fc5f97c6)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:18.800507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e345719-24b1-4fb2-833a-131165b49496 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:18.802459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 197/414] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:18 2022] GET /identity/v3/projects/e314473e6b1442dfa9c0bcc6fc5f97c6 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:18.808764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad7ad0-2050-43bd-9160-b0aeef39b49c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:18.915606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:18.938658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad7ad0-2050-43bd-9160-b0aeef39b49c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:18.974013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171fd24-cd6f-40a6-9524-80025b75d813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:18.981125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171fd24-cd6f-40a6-9524-80025b75d813 None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:18.981125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171fd24-cd6f-40a6-9524-80025b75d813 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:18.981125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171fd24-cd6f-40a6-9524-80025b75d813 None admin] PATH_INFO: `/v3/projects/e314473e6b1442dfa9c0bcc6fc5f97c6/groups/2a0fbe291c304fe0bd92d0dec4ec1ecc/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:18.981125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171fd24-cd6f-40a6-9524-80025b75d813 None admin] RBAC: Authorizing `identity:create_grant(project_id=e314473e6b1442dfa9c0bcc6fc5f97c6, group_id=2a0fbe291c304fe0bd92d0dec4ec1ecc, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:18.981125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171fd24-cd6f-40a6-9524-80025b75d813 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:19.093195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 218/415] 10.222.0.46 () {58 vars in 1442 bytes} [Fri Sep 9 12:37:18 2022] PUT /identity/v3/projects/e314473e6b1442dfa9c0bcc6fc5f97c6/groups/2a0fbe291c304fe0bd92d0dec4ec1ecc/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:37:19.471656 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7928b0f-d8e9-4a8e-ba80-2b3cc5390fa9 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:19.472643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7928b0f-d8e9-4a8e-ba80-2b3cc5390fa9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:19.473121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7928b0f-d8e9-4a8e-ba80-2b3cc5390fa9 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:19.475082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 198/416] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:19.481964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65211233-56fb-4f12-a4de-b4420404d208 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:19.482472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65211233-56fb-4f12-a4de-b4420404d208 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:19.482584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65211233-56fb-4f12-a4de-b4420404d208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:19.504539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-65211233-56fb-4f12-a4de-b4420404d208 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:19.505725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-65211233-56fb-4f12-a4de-b4420404d208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:19.597061 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 219/417] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:19.604672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75009a85-fbe7-493b-8515-4ed638730993 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:19.605030 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75009a85-fbe7-493b-8515-4ed638730993 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:19.605374 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75009a85-fbe7-493b-8515-4ed638730993 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:19.606954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 199/418] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:19.622084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2d2d5-ba37-40f7-8414-be6c0f96309e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:19.622659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2d2d5-ba37-40f7-8414-be6c0f96309e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:19.623108 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2d2d5-ba37-40f7-8414-be6c0f96309e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:19.645832 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-dbf2d2d5-ba37-40f7-8414-be6c0f96309e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:19.647311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-dbf2d2d5-ba37-40f7-8414-be6c0f96309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:19.713909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:19.716967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 220/419] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:19.870516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75009a85-fbe7-493b-8515-4ed638730993 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:20.012115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75009a85-fbe7-493b-8515-4ed638730993 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:20.026124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:20.053419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beccbd2-b9e9-43ac-b4ad-e5b93d3fdd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:20.054045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beccbd2-b9e9-43ac-b4ad-e5b93d3fdd14 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:20.057043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beccbd2-b9e9-43ac-b4ad-e5b93d3fdd14 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:20.057373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beccbd2-b9e9-43ac-b4ad-e5b93d3fdd14 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:20.072909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beccbd2-b9e9-43ac-b4ad-e5b93d3fdd14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-2beccbd2-b9e9-43ac-b4ad-e5b93d3fdd14 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:20.077703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:20.078606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:20.079573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:20.079573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:20.079573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:20.079573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:20.079573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:20.079573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:20.079573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:20.079573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:20.079573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:20.079573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:20.079573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:20.079573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:20.079573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 200/420] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:37:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:20.084761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2d2d5-ba37-40f7-8414-be6c0f96309e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:20.097960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beccbd2-b9e9-43ac-b4ad-e5b93d3fdd14 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:20.201989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beccbd2-b9e9-43ac-b4ad-e5b93d3fdd14 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:20.236452 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052bd80-138a-456e-8fd8-b524e13fff78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:20.237092 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052bd80-138a-456e-8fd8-b524e13fff78 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:20.237380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052bd80-138a-456e-8fd8-b524e13fff78 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:20.237663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052bd80-138a-456e-8fd8-b524e13fff78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:20.248658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052bd80-138a-456e-8fd8-b524e13fff78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-c052bd80-138a-456e-8fd8-b524e13fff78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:20.253897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:20.254871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:20.256215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:20.256215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:20.256215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:20.256215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:20.256215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:20.256215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:20.256215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:20.256215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:20.256215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:20.256215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:20.256215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:20.256215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:20.256215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 201/421] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:20.262467 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052bd80-138a-456e-8fd8-b524e13fff78 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:20.294872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:20.324399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2d2d5-ba37-40f7-8414-be6c0f96309e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:20.367773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052bd80-138a-456e-8fd8-b524e13fff78 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:20.404666 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71b861-a3bb-4bf0-b010-32898212d03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:20.405210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71b861-a3bb-4bf0-b010-32898212d03f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:20.405498 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71b861-a3bb-4bf0-b010-32898212d03f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:20.405837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71b861-a3bb-4bf0-b010-32898212d03f None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:20.406535 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71b861-a3bb-4bf0-b010-32898212d03f None admin] RBAC: Adding query filter params (name=member) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:20.406899 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71b861-a3bb-4bf0-b010-32898212d03f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:20.408542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71b861-a3bb-4bf0-b010-32898212d03f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:20.417766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 202/422] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:37:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:20.423511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71b861-a3bb-4bf0-b010-32898212d03f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:20.467720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ed237-2417-4c35-b8fd-ffcf93d56f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:20.468353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ed237-2417-4c35-b8fd-ffcf93d56f62 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:20.468608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ed237-2417-4c35-b8fd-ffcf93d56f62 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:20.468877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ed237-2417-4c35-b8fd-ffcf93d56f62 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:20.469544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ed237-2417-4c35-b8fd-ffcf93d56f62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:20.469860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ed237-2417-4c35-b8fd-ffcf93d56f62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:20.471226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ed237-2417-4c35-b8fd-ffcf93d56f62 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:20.499634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 221/423] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:20.505149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ed237-2417-4c35-b8fd-ffcf93d56f62 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:20.527388 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:20.531026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71b861-a3bb-4bf0-b010-32898212d03f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:20.567979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c7fad-e2ee-477e-8d90-1c2b017a88c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:20.568598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c7fad-e2ee-477e-8d90-1c2b017a88c5 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:20.568890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c7fad-e2ee-477e-8d90-1c2b017a88c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:20.569170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c7fad-e2ee-477e-8d90-1c2b017a88c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:20.640115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c7fad-e2ee-477e-8d90-1c2b017a88c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:20.641593 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c7fad-e2ee-477e-8d90-1c2b017a88c5 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:20.678950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:20.741501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:20.748215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 203/424] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:20.748707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ed237-2417-4c35-b8fd-ffcf93d56f62 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:20.754118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c7fad-e2ee-477e-8d90-1c2b017a88c5 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:20.807352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf132a0-6b49-4a7f-9114-b19050f3a89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:20.807971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf132a0-6b49-4a7f-9114-b19050f3a89a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:20.808248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf132a0-6b49-4a7f-9114-b19050f3a89a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:20.808521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf132a0-6b49-4a7f-9114-b19050f3a89a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:20.858409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c7fad-e2ee-477e-8d90-1c2b017a88c5 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:20.879970 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf132a0-6b49-4a7f-9114-b19050f3a89a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:20.881490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf132a0-6b49-4a7f-9114-b19050f3a89a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:20.915192 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909a0f5-3e09-42c5-bdc2-2543f8b516b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:20.915842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909a0f5-3e09-42c5-bdc2-2543f8b516b1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:20.916126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909a0f5-3e09-42c5-bdc2-2543f8b516b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:20.916401 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909a0f5-3e09-42c5-bdc2-2543f8b516b1 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:20.917576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909a0f5-3e09-42c5-bdc2-2543f8b516b1 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:20.919066 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909a0f5-3e09-42c5-bdc2-2543f8b516b1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:20.920696 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 204/425] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:20 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:20.927440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909a0f5-3e09-42c5-bdc2-2543f8b516b1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:20.962936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:20.985954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 222/426] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:20.991120 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf132a0-6b49-4a7f-9114-b19050f3a89a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:21.035653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:21.090868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909a0f5-3e09-42c5-bdc2-2543f8b516b1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:21.142829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787516ed-80f3-49f7-8d42-95513fa77f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:21.143722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787516ed-80f3-49f7-8d42-95513fa77f7b None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:21.144153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787516ed-80f3-49f7-8d42-95513fa77f7b None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:21.144580 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787516ed-80f3-49f7-8d42-95513fa77f7b None admin] PATH_INFO: `/v3/groups/940b14c2156d43a9916a8e5b4311fb74` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:21.145989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787516ed-80f3-49f7-8d42-95513fa77f7b None admin] RBAC: Authorizing `identity:get_group(group_id=940b14c2156d43a9916a8e5b4311fb74)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:21.148134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787516ed-80f3-49f7-8d42-95513fa77f7b None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:21.150337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 205/427] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:20 2022] GET /identity/v3/groups/940b14c2156d43a9916a8e5b4311fb74 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:21.159432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787516ed-80f3-49f7-8d42-95513fa77f7b None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:21.221455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf132a0-6b49-4a7f-9114-b19050f3a89a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:21.282674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d27e3-b8b2-4a3a-8d38-4549a5441f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:21.283302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d27e3-b8b2-4a3a-8d38-4549a5441f44 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:21.283573 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d27e3-b8b2-4a3a-8d38-4549a5441f44 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:21.283839 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d27e3-b8b2-4a3a-8d38-4549a5441f44 None admin] PATH_INFO: `/v3/projects/8b0387feea344c35a5907f20e962c357` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:21.284976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d27e3-b8b2-4a3a-8d38-4549a5441f44 None admin] RBAC: Authorizing `identity:get_project(project_id=8b0387feea344c35a5907f20e962c357)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:21.286401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d27e3-b8b2-4a3a-8d38-4549a5441f44 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:21.290505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 223/428] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:20 2022] GET /identity/v3/projects/8b0387feea344c35a5907f20e962c357 => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:21.293844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d27e3-b8b2-4a3a-8d38-4549a5441f44 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:21.334681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787516ed-80f3-49f7-8d42-95513fa77f7b None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:21.340251 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:21.363696 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:21.373582 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644bfec-c124-4384-ba19-50ecc53c663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:21.374197 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644bfec-c124-4384-ba19-50ecc53c663f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:21.374652 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644bfec-c124-4384-ba19-50ecc53c663f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:21.374924 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644bfec-c124-4384-ba19-50ecc53c663f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:21.376167 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644bfec-c124-4384-ba19-50ecc53c663f None admin] RBAC: Adding query filter params (group.id=940b14c2156d43a9916a8e5b4311fb74, role.id=43667d074eda4143bf3b7f721e83810c, scope.project.id=8b0387feea344c35a5907f20e962c357) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:21.376612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644bfec-c124-4384-ba19-50ecc53c663f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:21.378098 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644bfec-c124-4384-ba19-50ecc53c663f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:21.393055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 206/429] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 12:37:21 2022] GET /identity/v3/role_assignments?group.id=940b14c2156d43a9916a8e5b4311fb74&scope.project.id=8b0387feea344c35a5907f20e962c357&role.id=43667d074eda4143bf3b7f721e83810c => generated 611 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:21.393055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1f69f-e8ad-499c-9292-e93b2cd7f083 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:21.393055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1f69f-e8ad-499c-9292-e93b2cd7f083 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:21.393055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1f69f-e8ad-499c-9292-e93b2cd7f083 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:21.393055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 207/430] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:21.393055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4217c3-4c57-44e4-b71a-e2bb5130414f None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:21.393612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4217c3-4c57-44e4-b71a-e2bb5130414f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:21.393612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4217c3-4c57-44e4-b71a-e2bb5130414f None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:21.395509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 208/431] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:21.400627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2bb3b-90c9-4c58-90b8-965eb59125ab None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:21.400904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2bb3b-90c9-4c58-90b8-965eb59125ab None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:21.401175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2bb3b-90c9-4c58-90b8-965eb59125ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:21.425525 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-bcb2bb3b-90c9-4c58-90b8-965eb59125ab None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:21.426983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-bcb2bb3b-90c9-4c58-90b8-965eb59125ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:21.427418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d27e3-b8b2-4a3a-8d38-4549a5441f44 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:21.467047 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705b0a1-dbf0-402a-8ce6-bd31d0b7ad64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:21.467801 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705b0a1-dbf0-402a-8ce6-bd31d0b7ad64 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:21.468075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705b0a1-dbf0-402a-8ce6-bd31d0b7ad64 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:21.468341 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705b0a1-dbf0-402a-8ce6-bd31d0b7ad64 None admin] PATH_INFO: `/v3/groups/940b14c2156d43a9916a8e5b4311fb74` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:21.469490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705b0a1-dbf0-402a-8ce6-bd31d0b7ad64 None admin] RBAC: Authorizing `identity:get_group(group_id=940b14c2156d43a9916a8e5b4311fb74)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:21.470999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705b0a1-dbf0-402a-8ce6-bd31d0b7ad64 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:21.472688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 224/432] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:21 2022] GET /identity/v3/groups/940b14c2156d43a9916a8e5b4311fb74 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:21.474022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0220df-cf2d-406a-b0d9-f65554c155e6 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:21.474298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0220df-cf2d-406a-b0d9-f65554c155e6 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:21.474619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0220df-cf2d-406a-b0d9-f65554c155e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:21.495045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-3e0220df-cf2d-406a-b0d9-f65554c155e6 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:21.496247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-3e0220df-cf2d-406a-b0d9-f65554c155e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:21.514644 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 209/433] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:21.515670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2bb3b-90c9-4c58-90b8-965eb59125ab None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:21.616659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 225/434] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:21.675200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2bb3b-90c9-4c58-90b8-965eb59125ab None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:21.702078 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:21.716197 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aaead2-391b-4c2f-81d7-7e3fc918f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:21.717170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aaead2-391b-4c2f-81d7-7e3fc918f1d8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:21.717593 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aaead2-391b-4c2f-81d7-7e3fc918f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:21.718006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aaead2-391b-4c2f-81d7-7e3fc918f1d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:21.719041 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aaead2-391b-4c2f-81d7-7e3fc918f1d8 None admin] RBAC: Adding query filter params (group.id=940b14c2156d43a9916a8e5b4311fb74, role.id=96271a2245c144d4b43dea66f18074e8, scope.project.id=8b0387feea344c35a5907f20e962c357) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:21.719525 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aaead2-391b-4c2f-81d7-7e3fc918f1d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:21.721359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aaead2-391b-4c2f-81d7-7e3fc918f1d8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:21.729440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 210/435] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 12:37:21 2022] GET /identity/v3/role_assignments?group.id=940b14c2156d43a9916a8e5b4311fb74&scope.project.id=8b0387feea344c35a5907f20e962c357&role.id=96271a2245c144d4b43dea66f18074e8 => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:21.811694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0220df-cf2d-406a-b0d9-f65554c155e6 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:21.842936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:21.857718 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506a8ef-8058-4192-bbbf-2deb2a93cac9 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:21.858371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506a8ef-8058-4192-bbbf-2deb2a93cac9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:21.858872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506a8ef-8058-4192-bbbf-2deb2a93cac9 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:21.860779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 211/436] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:21.866045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ac88a-834e-457a-a8df-813a94f62c58 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:21.866337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ac88a-834e-457a-a8df-813a94f62c58 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:21.866651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ac88a-834e-457a-a8df-813a94f62c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:21.903474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-130ac88a-834e-457a-a8df-813a94f62c58 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:21.905068 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-130ac88a-834e-457a-a8df-813a94f62c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:21.935168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0220df-cf2d-406a-b0d9-f65554c155e6 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:21.977498 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05619ba6-9c42-4699-905a-5645e7336606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:21.978127 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05619ba6-9c42-4699-905a-5645e7336606 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:21.978403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05619ba6-9c42-4699-905a-5645e7336606 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:21.978719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05619ba6-9c42-4699-905a-5645e7336606 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:21.989315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 212/437] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:21.990418 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ac88a-834e-457a-a8df-813a94f62c58 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:21.991559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05619ba6-9c42-4699-905a-5645e7336606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-05619ba6-9c42-4699-905a-5645e7336606 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:21.997591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:21.998513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:21.999336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:21.999336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:21.999336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:21.999336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:21.999336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:21.999336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:21.999336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:21.999336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:21.999336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:21.999336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:21.999336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:21.999336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:21.999336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 226/438] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:37:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:22.006182 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05619ba6-9c42-4699-905a-5645e7336606 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:22.131154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ac88a-834e-457a-a8df-813a94f62c58 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:22.143892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:22.154859 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:22.165370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05619ba6-9c42-4699-905a-5645e7336606 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:22.182628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea76d3d-8e3c-4205-88f7-21b8057754c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:22.183209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea76d3d-8e3c-4205-88f7-21b8057754c4 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:22.183456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea76d3d-8e3c-4205-88f7-21b8057754c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:22.183724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea76d3d-8e3c-4205-88f7-21b8057754c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:22.195708 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea76d3d-8e3c-4205-88f7-21b8057754c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-2ea76d3d-8e3c-4205-88f7-21b8057754c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:22.201119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:22.202010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:22.202858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:22.202858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:22.202858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:22.202858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:22.202858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:22.202858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:22.202858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:22.202858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:22.202858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:22.202858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:22.202858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 12:37:22.202858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:22.202858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 213/439] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:22.210033 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea76d3d-8e3c-4205-88f7-21b8057754c4 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:22.219342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bf205-742a-407d-95e9-703afafe5b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:22.220230 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bf205-742a-407d-95e9-703afafe5b17 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:22.220720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bf205-742a-407d-95e9-703afafe5b17 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:22.221244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bf205-742a-407d-95e9-703afafe5b17 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:22.222111 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bf205-742a-407d-95e9-703afafe5b17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:22.223595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bf205-742a-407d-95e9-703afafe5b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:22.225158 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bf205-742a-407d-95e9-703afafe5b17 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:22.234461 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 227/440] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:22.240504 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bf205-742a-407d-95e9-703afafe5b17 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:22.326853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea76d3d-8e3c-4205-88f7-21b8057754c4 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:22.363043 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bf205-742a-407d-95e9-703afafe5b17 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:22.366586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3dceb-e693-4fa8-8cba-e20ddbd4bdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:22.367971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3dceb-e693-4fa8-8cba-e20ddbd4bdb6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:22.368250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3dceb-e693-4fa8-8cba-e20ddbd4bdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:22.368520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3dceb-e693-4fa8-8cba-e20ddbd4bdb6 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:22.369176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3dceb-e693-4fa8-8cba-e20ddbd4bdb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:22.369476 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3dceb-e693-4fa8-8cba-e20ddbd4bdb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:22.370982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3dceb-e693-4fa8-8cba-e20ddbd4bdb6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:22.379229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 214/441] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:37:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:22.380173 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3dceb-e693-4fa8-8cba-e20ddbd4bdb6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:22.399381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab12d54-f22c-47ce-b424-053f50f9976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:22.399981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab12d54-f22c-47ce-b424-053f50f9976e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:22.400256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab12d54-f22c-47ce-b424-053f50f9976e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:22.400525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab12d54-f22c-47ce-b424-053f50f9976e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:22.418664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:22.419656 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:22.465169 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab12d54-f22c-47ce-b424-053f50f9976e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:22.466618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab12d54-f22c-47ce-b424-053f50f9976e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:22.475879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3dceb-e693-4fa8-8cba-e20ddbd4bdb6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:22.509471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c0317-f81f-438e-8bc3-24581a54b9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:22.510020 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c0317-f81f-438e-8bc3-24581a54b9c2 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:22.510296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c0317-f81f-438e-8bc3-24581a54b9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:22.510613 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c0317-f81f-438e-8bc3-24581a54b9c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:22.520221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c0317-f81f-438e-8bc3-24581a54b9c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-509c0317-f81f-438e-8bc3-24581a54b9c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:22.525741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:22.526808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:22.527614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:22.527614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:22.527614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:22.527614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:22.527614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:22.527614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:22.527614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:22.527614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:22.527614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:22.527614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:22.527614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:37:22.527614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:22.527614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 215/442] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:37:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:22.528276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c0317-f81f-438e-8bc3-24581a54b9c2 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:22.563897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 228/443] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 12:37:22.564721 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab12d54-f22c-47ce-b424-053f50f9976e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:22.613188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:22.627751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:22.631438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c0317-f81f-438e-8bc3-24581a54b9c2 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:22.660385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab12d54-f22c-47ce-b424-053f50f9976e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:22.664700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e041d-ed9b-4887-9f88-e5e3ea6b6e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:22.665300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e041d-ed9b-4887-9f88-e5e3ea6b6e90 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:22.665582 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e041d-ed9b-4887-9f88-e5e3ea6b6e90 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:22.665855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e041d-ed9b-4887-9f88-e5e3ea6b6e90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:22.694278 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a1f9a-5d50-437f-b72a-e711d7ee014f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:22.695430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a1f9a-5d50-437f-b72a-e711d7ee014f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:22.695872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a1f9a-5d50-437f-b72a-e711d7ee014f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:22.696157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a1f9a-5d50-437f-b72a-e711d7ee014f None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:22.696811 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a1f9a-5d50-437f-b72a-e711d7ee014f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:22.697108 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a1f9a-5d50-437f-b72a-e711d7ee014f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:22.698456 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a1f9a-5d50-437f-b72a-e711d7ee014f None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:22.706272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 229/444] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:22.707082 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a1f9a-5d50-437f-b72a-e711d7ee014f None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:22.738332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e041d-ed9b-4887-9f88-e5e3ea6b6e90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:22.739811 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e041d-ed9b-4887-9f88-e5e3ea6b6e90 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:22.809051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a1f9a-5d50-437f-b72a-e711d7ee014f None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:22.821687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:22.830952 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:22.834928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 216/445] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:22.836364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e041d-ed9b-4887-9f88-e5e3ea6b6e90 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:22.843307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c222162-ab4c-4771-bad2-c142f0369b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:22.843931 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c222162-ab4c-4771-bad2-c142f0369b3c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:22.844208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c222162-ab4c-4771-bad2-c142f0369b3c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:22.844478 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c222162-ab4c-4771-bad2-c142f0369b3c None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:22.845585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c222162-ab4c-4771-bad2-c142f0369b3c None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:22.847042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c222162-ab4c-4771-bad2-c142f0369b3c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:22.848625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 230/446] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:22 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:22.849362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c222162-ab4c-4771-bad2-c142f0369b3c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:22.932381 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e041d-ed9b-4887-9f88-e5e3ea6b6e90 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:22.946398 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c222162-ab4c-4771-bad2-c142f0369b3c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:22.967323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d7eb8-4dbd-42b1-82ef-3b885f138134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:22.967916 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7eb8-4dbd-42b1-82ef-3b885f138134 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:22.968187 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7eb8-4dbd-42b1-82ef-3b885f138134 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:22.968459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7eb8-4dbd-42b1-82ef-3b885f138134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:22.978290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661125bc-5531-4e71-8798-9c61187e2ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:22.978956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661125bc-5531-4e71-8798-9c61187e2ca5 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:22.979231 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661125bc-5531-4e71-8798-9c61187e2ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:22.979501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661125bc-5531-4e71-8798-9c61187e2ca5 None admin] PATH_INFO: `/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:22.980591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661125bc-5531-4e71-8798-9c61187e2ca5 None admin] RBAC: Authorizing `identity:get_project(project_id=3cab176a56624fcbb4fd9cf5ad72ab30)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:22.981995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661125bc-5531-4e71-8798-9c61187e2ca5 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:22.983621 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 231/447] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:22 2022] GET /identity/v3/projects/3cab176a56624fcbb4fd9cf5ad72ab30 => generated 302 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:22.984359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661125bc-5531-4e71-8798-9c61187e2ca5 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:23.031499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:23.033204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:23.035131 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d7eb8-4dbd-42b1-82ef-3b885f138134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:23.036638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d7eb8-4dbd-42b1-82ef-3b885f138134 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:23.083042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661125bc-5531-4e71-8798-9c61187e2ca5 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:23.117817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e6bcf-2b75-4bef-8123-e1f9b5b20cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:23.118712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e6bcf-2b75-4bef-8123-e1f9b5b20cfd None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:23.118991 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e6bcf-2b75-4bef-8123-e1f9b5b20cfd None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:23.119275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e6bcf-2b75-4bef-8123-e1f9b5b20cfd None admin] PATH_INFO: `/v3/groups/940b14c2156d43a9916a8e5b4311fb74` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:23.120393 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e6bcf-2b75-4bef-8123-e1f9b5b20cfd None admin] RBAC: Authorizing `identity:get_group(group_id=940b14c2156d43a9916a8e5b4311fb74)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:23.121819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e6bcf-2b75-4bef-8123-e1f9b5b20cfd None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:23.123411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 232/448] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:22 2022] GET /identity/v3/groups/940b14c2156d43a9916a8e5b4311fb74 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:23.124169 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e6bcf-2b75-4bef-8123-e1f9b5b20cfd None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:23.140970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 217/449] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:23.141801 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d7eb8-4dbd-42b1-82ef-3b885f138134 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:23.225041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e6bcf-2b75-4bef-8123-e1f9b5b20cfd None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:23.235851 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:23.239617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d7eb8-4dbd-42b1-82ef-3b885f138134 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:23.240286 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:23.256508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c79007-d392-4368-addd-b1e04dc73649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:23.257112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c79007-d392-4368-addd-b1e04dc73649 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:23.257912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c79007-d392-4368-addd-b1e04dc73649 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:23.258184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c79007-d392-4368-addd-b1e04dc73649 None admin] PATH_INFO: `/v3/groups/940b14c2156d43a9916a8e5b4311fb74` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:23.259387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c79007-d392-4368-addd-b1e04dc73649 None admin] RBAC: Authorizing `identity:get_group(group_id=940b14c2156d43a9916a8e5b4311fb74)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:23.260789 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c79007-d392-4368-addd-b1e04dc73649 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:23.262318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 233/450] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:23 2022] GET /identity/v3/groups/940b14c2156d43a9916a8e5b4311fb74 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:23.263097 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c79007-d392-4368-addd-b1e04dc73649 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:23.272987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d0934-9902-408b-97bd-5f744b4d7c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:23.274275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d0934-9902-408b-97bd-5f744b4d7c8f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:23.274587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d0934-9902-408b-97bd-5f744b4d7c8f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:23.274860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d0934-9902-408b-97bd-5f744b4d7c8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:23.275607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d0934-9902-408b-97bd-5f744b4d7c8f None admin] RBAC: Adding query filter params (group.id=940b14c2156d43a9916a8e5b4311fb74, role.id=96271a2245c144d4b43dea66f18074e8, scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:23.275920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d0934-9902-408b-97bd-5f744b4d7c8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:23.277370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d0934-9902-408b-97bd-5f744b4d7c8f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:23.283205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 218/451] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 12:37:23 2022] GET /identity/v3/role_assignments?group.id=940b14c2156d43a9916a8e5b4311fb74&scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30&role.id=96271a2245c144d4b43dea66f18074e8 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:23.283977 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d0934-9902-408b-97bd-5f744b4d7c8f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:23.376835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c79007-d392-4368-addd-b1e04dc73649 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:23.398007 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d0934-9902-408b-97bd-5f744b4d7c8f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:23.412698 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e4ff2-caa9-4608-845c-b24fecbb7b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:23.413350 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e4ff2-caa9-4608-845c-b24fecbb7b04 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:23.413631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e4ff2-caa9-4608-845c-b24fecbb7b04 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:23.413957 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e4ff2-caa9-4608-845c-b24fecbb7b04 None admin] PATH_INFO: `/v3/projects/e314473e6b1442dfa9c0bcc6fc5f97c6` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:23.415210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e4ff2-caa9-4608-845c-b24fecbb7b04 None admin] RBAC: Authorizing `identity:get_project(project_id=e314473e6b1442dfa9c0bcc6fc5f97c6)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:23.416656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e4ff2-caa9-4608-845c-b24fecbb7b04 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:23.418336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 234/452] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:37:23 2022] GET /identity/v3/projects/e314473e6b1442dfa9c0bcc6fc5f97c6 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:23.424000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e4ff2-caa9-4608-845c-b24fecbb7b04 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:23.437702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c0cb5-4e37-424f-9d1e-a7744250058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:23.438312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c0cb5-4e37-424f-9d1e-a7744250058a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:23.438627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c0cb5-4e37-424f-9d1e-a7744250058a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:23.438896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c0cb5-4e37-424f-9d1e-a7744250058a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:23.439623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c0cb5-4e37-424f-9d1e-a7744250058a None admin] RBAC: Adding query filter params (group.id=940b14c2156d43a9916a8e5b4311fb74, role.id=43667d074eda4143bf3b7f721e83810c, scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:23.439925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c0cb5-4e37-424f-9d1e-a7744250058a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:23.441637 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c0cb5-4e37-424f-9d1e-a7744250058a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:23.449772 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 219/453] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 12:37:23 2022] GET /identity/v3/role_assignments?group.id=940b14c2156d43a9916a8e5b4311fb74&scope.project.id=3cab176a56624fcbb4fd9cf5ad72ab30&role.id=43667d074eda4143bf3b7f721e83810c => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:23.485585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:23.544247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e4ff2-caa9-4608-845c-b24fecbb7b04 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:23.580808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669aa65b-114b-4df3-b5ed-61a5a8b4c467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:23.581457 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669aa65b-114b-4df3-b5ed-61a5a8b4c467 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:23.581735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669aa65b-114b-4df3-b5ed-61a5a8b4c467 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:23.582007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669aa65b-114b-4df3-b5ed-61a5a8b4c467 None admin] PATH_INFO: `/v3/groups/2a0fbe291c304fe0bd92d0dec4ec1ecc` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:23.583208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669aa65b-114b-4df3-b5ed-61a5a8b4c467 None admin] RBAC: Authorizing `identity:get_group(group_id=2a0fbe291c304fe0bd92d0dec4ec1ecc)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:23.584638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669aa65b-114b-4df3-b5ed-61a5a8b4c467 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:23.586240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 235/454] 10.222.0.46 () {56 vars in 1180 bytes} [Fri Sep 9 12:37:23 2022] GET /identity/v3/groups/2a0fbe291c304fe0bd92d0dec4ec1ecc => generated 229 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:23.592515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c0cb5-4e37-424f-9d1e-a7744250058a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:23.621594 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:23.695547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c0cb5-4e37-424f-9d1e-a7744250058a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:23.729494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef1b4a-08f8-4609-ac30-6b4750d62555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:23.730176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef1b4a-08f8-4609-ac30-6b4750d62555 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:23.730466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef1b4a-08f8-4609-ac30-6b4750d62555 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:23.730777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef1b4a-08f8-4609-ac30-6b4750d62555 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:23.731502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef1b4a-08f8-4609-ac30-6b4750d62555 None admin] RBAC: Adding query filter params (group.id=2a0fbe291c304fe0bd92d0dec4ec1ecc, role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=e314473e6b1442dfa9c0bcc6fc5f97c6) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:23.731813 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef1b4a-08f8-4609-ac30-6b4750d62555 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:23.733263 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef1b4a-08f8-4609-ac30-6b4750d62555 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:23.739108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 220/455] 10.222.0.46 () {56 vars in 1376 bytes} [Fri Sep 9 12:37:23 2022] GET /identity/v3/role_assignments?group.id=2a0fbe291c304fe0bd92d0dec4ec1ecc&scope.project.id=e314473e6b1442dfa9c0bcc6fc5f97c6&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:26.391259 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751cdb8-f9d7-463d-93e5-e16325416763 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:26.392219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751cdb8-f9d7-463d-93e5-e16325416763 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:26.392691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751cdb8-f9d7-463d-93e5-e16325416763 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:26.394449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 236/456] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:26.400205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cee67-4aa6-493b-a63b-9350228671b0 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:26.400660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cee67-4aa6-493b-a63b-9350228671b0 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:26.400735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cee67-4aa6-493b-a63b-9350228671b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:26.428408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-553cee67-4aa6-493b-a63b-9350228671b0 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:26.429648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-553cee67-4aa6-493b-a63b-9350228671b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:26.512937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 221/457] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:37:26.682265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbd857-7fdb-45df-b50d-25219cbe79bc None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:26.682571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbd857-7fdb-45df-b50d-25219cbe79bc None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:26.682804 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbd857-7fdb-45df-b50d-25219cbe79bc None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:26.684280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 237/458] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:26.689704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe48279-eae3-44c0-8461-5cc2dd92289e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:26.689995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe48279-eae3-44c0-8461-5cc2dd92289e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:26.690270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe48279-eae3-44c0-8461-5cc2dd92289e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:26.719574 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-bfe48279-eae3-44c0-8461-5cc2dd92289e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:26.720807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-bfe48279-eae3-44c0-8461-5cc2dd92289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:26.738229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:26.801493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 222/459] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:26.961625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbd857-7fdb-45df-b50d-25219cbe79bc None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:27.001323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11a54f-845a-44e7-a895-196f3010b29d None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:27.001950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11a54f-845a-44e7-a895-196f3010b29d None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:27.002435 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11a54f-845a-44e7-a895-196f3010b29d None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:27.004536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 223/460] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:27.011421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20131294-c433-4a6e-8014-1094d71e2411 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:27.012058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20131294-c433-4a6e-8014-1094d71e2411 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:27.012525 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20131294-c433-4a6e-8014-1094d71e2411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:27.045249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-20131294-c433-4a6e-8014-1094d71e2411 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:27.047495 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-20131294-c433-4a6e-8014-1094d71e2411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:27.144808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbd857-7fdb-45df-b50d-25219cbe79bc None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:27.159222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 224/461] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:37:27.160696 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de2426-3e6f-4b0e-b7e3-50d05f0731b0 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:27.160985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de2426-3e6f-4b0e-b7e3-50d05f0731b0 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:27.161326 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de2426-3e6f-4b0e-b7e3-50d05f0731b0 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:27.162609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 225/462] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:27.163301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de2426-3e6f-4b0e-b7e3-50d05f0731b0 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:27.170108 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:27.189803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a2da2-152d-47f7-9d88-418cde6aaada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:27.190769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a2da2-152d-47f7-9d88-418cde6aaada None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:27.191794 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a2da2-152d-47f7-9d88-418cde6aaada None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:27.192292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a2da2-152d-47f7-9d88-418cde6aaada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:27.208202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a2da2-152d-47f7-9d88-418cde6aaada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:27.209975 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a2da2-152d-47f7-9d88-418cde6aaada None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-f92a2da2-152d-47f7-9d88-418cde6aaada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:27.229830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:27.230833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:27.231885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:27.231885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:27.231885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:27.233793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 238/463] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:27.238992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa6505-df77-4ca5-a090-3b2e5111d8bf None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:27.238992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa6505-df77-4ca5-a090-3b2e5111d8bf None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:27.238992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa6505-df77-4ca5-a090-3b2e5111d8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:27.261233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-68fa6505-df77-4ca5-a090-3b2e5111d8bf None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:27.262858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-68fa6505-df77-4ca5-a090-3b2e5111d8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:27.264692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:27.284917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de2426-3e6f-4b0e-b7e3-50d05f0731b0 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:27.321248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816156d-fb59-4dc6-9d99-b9dc62f9ec51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:27.321868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816156d-fb59-4dc6-9d99-b9dc62f9ec51 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:27.322151 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816156d-fb59-4dc6-9d99-b9dc62f9ec51 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:27.322425 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816156d-fb59-4dc6-9d99-b9dc62f9ec51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:27.332842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816156d-fb59-4dc6-9d99-b9dc62f9ec51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:27.334295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816156d-fb59-4dc6-9d99-b9dc62f9ec51 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:27.341467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 239/464] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:27.342687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa6505-df77-4ca5-a090-3b2e5111d8bf None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-7816156d-fb59-4dc6-9d99-b9dc62f9ec51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:27.357301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:27.358253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:27.359215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:27.359215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:27.359215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:27.366924 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 226/465] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:27.373329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816156d-fb59-4dc6-9d99-b9dc62f9ec51 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:27.456911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa6505-df77-4ca5-a090-3b2e5111d8bf None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:27.470804 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:27.494655 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddace5-1558-4584-aeb0-7a4f28a4b5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:27.495597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddace5-1558-4584-aeb0-7a4f28a4b5fd None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:27.496073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddace5-1558-4584-aeb0-7a4f28a4b5fd None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:27.496576 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddace5-1558-4584-aeb0-7a4f28a4b5fd None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:27.497356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddace5-1558-4584-aeb0-7a4f28a4b5fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:27.497798 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddace5-1558-4584-aeb0-7a4f28a4b5fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:27.499530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddace5-1558-4584-aeb0-7a4f28a4b5fd None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:27.502220 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816156d-fb59-4dc6-9d99-b9dc62f9ec51 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:27.523599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 240/466] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:27.525019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddace5-1558-4584-aeb0-7a4f28a4b5fd None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:27.533770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:27.539302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f1a9f-2362-4d9d-9281-416a37c9209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:27.539946 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f1a9f-2362-4d9d-9281-416a37c9209a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:27.540228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f1a9f-2362-4d9d-9281-416a37c9209a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:27.540505 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f1a9f-2362-4d9d-9281-416a37c9209a None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:27.541158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f1a9f-2362-4d9d-9281-416a37c9209a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:27.541511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f1a9f-2362-4d9d-9281-416a37c9209a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:27.542942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f1a9f-2362-4d9d-9281-416a37c9209a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:27.561557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 227/467] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:27.561557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f1a9f-2362-4d9d-9281-416a37c9209a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:27.658675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddace5-1558-4584-aeb0-7a4f28a4b5fd None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:27.668790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f1a9f-2362-4d9d-9281-416a37c9209a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:27.693690 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a7b01-fab6-493b-b154-eeb9eef06aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:27.694321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a7b01-fab6-493b-b154-eeb9eef06aad None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:27.694628 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a7b01-fab6-493b-b154-eeb9eef06aad None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:27.695435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a7b01-fab6-493b-b154-eeb9eef06aad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:27.705371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a7b01-fab6-493b-b154-eeb9eef06aad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:27.706832 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a7b01-fab6-493b-b154-eeb9eef06aad None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:27.713131 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72cb40-cb83-4c60-a357-11424043e956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:27.714332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72cb40-cb83-4c60-a357-11424043e956 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:27.714800 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72cb40-cb83-4c60-a357-11424043e956 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:27.715239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72cb40-cb83-4c60-a357-11424043e956 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:27.716417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72cb40-cb83-4c60-a357-11424043e956 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:27.718302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72cb40-cb83-4c60-a357-11424043e956 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-355a7b01-fab6-493b-b154-eeb9eef06aad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:27.721741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:27.728933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:27.729958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:27.729958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:27.729958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:27.729958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 241/468] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:27.729958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a7b01-fab6-493b-b154-eeb9eef06aad None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:27.743699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:27.826004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a7b01-fab6-493b-b154-eeb9eef06aad None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:27.842558 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 228/469] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:37:27 2022] POST /identity/v3/users => generated 246 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:37:27.842558 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72cb40-cb83-4c60-a357-11424043e956 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:27.875544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c045ea-b9fd-439c-be48-a60cf1c1bcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:27.876292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c045ea-b9fd-439c-be48-a60cf1c1bcf6 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:27.876581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c045ea-b9fd-439c-be48-a60cf1c1bcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:27.876895 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c045ea-b9fd-439c-be48-a60cf1c1bcf6 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:27.877679 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c045ea-b9fd-439c-be48-a60cf1c1bcf6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:27.879351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c045ea-b9fd-439c-be48-a60cf1c1bcf6 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:27.904441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:27.960098 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72cb40-cb83-4c60-a357-11424043e956 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:27.974535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 242/470] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:37:27 2022] POST /identity/v3/users => generated 248 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:37:27.974535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c045ea-b9fd-439c-be48-a60cf1c1bcf6 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:28.016722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8373b-e5c1-4b8e-af1e-b95f3d072a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:28.017659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8373b-e5c1-4b8e-af1e-b95f3d072a4f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:28.017949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8373b-e5c1-4b8e-af1e-b95f3d072a4f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:28.018289 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8373b-e5c1-4b8e-af1e-b95f3d072a4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:28.031383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8373b-e5c1-4b8e-af1e-b95f3d072a4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:28.032938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8373b-e5c1-4b8e-af1e-b95f3d072a4f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-77a8373b-e5c1-4b8e-af1e-b95f3d072a4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:28.050566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:28.056228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:28.057134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:28.057134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:28.057134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:28.057134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 229/471] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:28.057134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffbfeb-1a80-4dbb-abc7-0c953fcd0d2e None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:28.057134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffbfeb-1a80-4dbb-abc7-0c953fcd0d2e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:28.057134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffbfeb-1a80-4dbb-abc7-0c953fcd0d2e None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:28.061170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 230/472] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:28.061703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffbfeb-1a80-4dbb-abc7-0c953fcd0d2e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:28.074159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:28.129590 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c045ea-b9fd-439c-be48-a60cf1c1bcf6 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:28.176925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc18b0a-6edc-4966-beb4-e471a5a36003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:28.177652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc18b0a-6edc-4966-beb4-e471a5a36003 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:28.177925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc18b0a-6edc-4966-beb4-e471a5a36003 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:28.178197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc18b0a-6edc-4966-beb4-e471a5a36003 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:28.178908 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc18b0a-6edc-4966-beb4-e471a5a36003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:28.179729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc18b0a-6edc-4966-beb4-e471a5a36003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:28.181118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc18b0a-6edc-4966-beb4-e471a5a36003 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:28.189018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:28.192831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffbfeb-1a80-4dbb-abc7-0c953fcd0d2e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:28.193604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 243/473] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:28.198184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc86637-71bf-474a-b0b6-b7fb9ab46d00 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:28.198184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc86637-71bf-474a-b0b6-b7fb9ab46d00 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:28.198184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc86637-71bf-474a-b0b6-b7fb9ab46d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:28.216081 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-cdc86637-71bf-474a-b0b6-b7fb9ab46d00 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:28.217282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-cdc86637-71bf-474a-b0b6-b7fb9ab46d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:28.250057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb678a5e-6a4c-4f54-9602-cbe5385f32a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:28.250707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb678a5e-6a4c-4f54-9602-cbe5385f32a8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:28.251433 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb678a5e-6a4c-4f54-9602-cbe5385f32a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:28.251712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb678a5e-6a4c-4f54-9602-cbe5385f32a8 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:28.252635 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb678a5e-6a4c-4f54-9602-cbe5385f32a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:28.252940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb678a5e-6a4c-4f54-9602-cbe5385f32a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:28.255180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb678a5e-6a4c-4f54-9602-cbe5385f32a8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:28.269763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 231/474] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:28.270632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb678a5e-6a4c-4f54-9602-cbe5385f32a8 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:28.305054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 244/475] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:28.310677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc86637-71bf-474a-b0b6-b7fb9ab46d00 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:28.378145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:28.425243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc86637-71bf-474a-b0b6-b7fb9ab46d00 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:28.464897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb678a5e-6a4c-4f54-9602-cbe5385f32a8 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:28.467740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35131e-0e42-4d32-828a-2bdf572aa853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:28.468346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35131e-0e42-4d32-828a-2bdf572aa853 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:28.468618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35131e-0e42-4d32-828a-2bdf572aa853 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:28.468875 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35131e-0e42-4d32-828a-2bdf572aa853 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:28.469516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f35131e-0e42-4d32-828a-2bdf572aa853 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:28.470963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f35131e-0e42-4d32-828a-2bdf572aa853 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:28.509642 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11499f31-7bc5-4bf9-ab58-e35fae30e116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:28.510289 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11499f31-7bc5-4bf9-ab58-e35fae30e116 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:28.510587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11499f31-7bc5-4bf9-ab58-e35fae30e116 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:28.510802 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11499f31-7bc5-4bf9-ab58-e35fae30e116 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:28.511453 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11499f31-7bc5-4bf9-ab58-e35fae30e116 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:28.512972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11499f31-7bc5-4bf9-ab58-e35fae30e116 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:28.578795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 245/476] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:37:28 2022] POST /identity/v3/users => generated 249 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:37:28.612833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 232/477] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:37:28 2022] POST /identity/v3/users => generated 248 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:37:28.716137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35131e-0e42-4d32-828a-2bdf572aa853 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:28.845107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35131e-0e42-4d32-828a-2bdf572aa853 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:28.863480 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:28.880932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3c885-c7b1-41ab-b7dd-7687732eba0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:28.881536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3c885-c7b1-41ab-b7dd-7687732eba0b None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:28.881813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3c885-c7b1-41ab-b7dd-7687732eba0b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:28.882082 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3c885-c7b1-41ab-b7dd-7687732eba0b None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:28.883049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3c885-c7b1-41ab-b7dd-7687732eba0b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:28.884584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3c885-c7b1-41ab-b7dd-7687732eba0b None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:28.932222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.sql.core [None req-e9a3c885-c7b1-41ab-b7dd-7687732eba0b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 12:37:28.932222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 12:37:28.932222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [parameters: {'id': '909eaeb9723f44ecbc27165fdfe3d9f3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 12:37:28.932222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88041) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-e9a3c885-c7b1-41ab-b7dd-7687732eba0b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application conn.query(q) Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application result.read() Sep 09 12:37:28.950819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 12:37:28.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application next(self.gen) Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application yield resource Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application next(self.gen) Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise self.value Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application session.commit() Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 12:37:28.953137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.session.flush() Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application compat.raise_( Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 12:37:28.954381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 12:37:28.955311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application conn.query(q) Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application result.read() Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 12:37:28.956115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [parameters: {'id': '909eaeb9723f44ecbc27165fdfe3d9f3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:28.957705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 12:37:28.960344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:28.965269 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 246/478] 10.222.0.46 () {60 vars in 1131 bytes} [Fri Sep 9 12:37:28 2022] POST /identity/v3/roles => generated 144 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:28.965427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11499f31-7bc5-4bf9-ab58-e35fae30e116 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:28.982986 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:29.085936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11499f31-7bc5-4bf9-ab58-e35fae30e116 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:29.129900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93395ab3-440b-4009-899f-bca9c850ae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:29.130556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93395ab3-440b-4009-899f-bca9c850ae15 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:29.130892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93395ab3-440b-4009-899f-bca9c850ae15 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:29.131168 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93395ab3-440b-4009-899f-bca9c850ae15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:29.145536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93395ab3-440b-4009-899f-bca9c850ae15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-93395ab3-440b-4009-899f-bca9c850ae15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:29.151308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:29.152215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:29.153035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:29.153035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:29.153035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:29.153035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:29.153035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:29.153035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:29.153035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:29.153035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:29.153035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:29.153035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:29.153035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 12:37:29.153035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:29.158711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 233/479] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:37:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:29.159959 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3c885-c7b1-41ab-b7dd-7687732eba0b None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:29.321874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3c885-c7b1-41ab-b7dd-7687732eba0b None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:29.376155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339715ba-0e19-4f41-972d-533099ff9d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:29.376760 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339715ba-0e19-4f41-972d-533099ff9d2d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:29.377034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339715ba-0e19-4f41-972d-533099ff9d2d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:29.377299 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339715ba-0e19-4f41-972d-533099ff9d2d None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:29.377961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339715ba-0e19-4f41-972d-533099ff9d2d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:29.378258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339715ba-0e19-4f41-972d-533099ff9d2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:29.379734 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339715ba-0e19-4f41-972d-533099ff9d2d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:29.394086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 247/480] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:30.513153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedab68f-73ed-419d-a324-2ab736dae452 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:30.513682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedab68f-73ed-419d-a324-2ab736dae452 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:30.513682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedab68f-73ed-419d-a324-2ab736dae452 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:30.515419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 234/481] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:30.520720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d202c36-2bee-43bf-bef1-a0e3eb9e437e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:30.521190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d202c36-2bee-43bf-bef1-a0e3eb9e437e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:30.521287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d202c36-2bee-43bf-bef1-a0e3eb9e437e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:30.551604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0d202c36-2bee-43bf-bef1-a0e3eb9e437e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:30.552819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0d202c36-2bee-43bf-bef1-a0e3eb9e437e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:30.592506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:30.628683 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 248/482] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:30.657034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7579e-b55d-4e74-b1b5-911bf1eb009f None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:30.657332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7579e-b55d-4e74-b1b5-911bf1eb009f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:30.657582 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7579e-b55d-4e74-b1b5-911bf1eb009f None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:30.659203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 235/483] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:30.672294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993777-eb02-4d94-b964-2d929cfff490 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:30.672580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993777-eb02-4d94-b964-2d929cfff490 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:30.672854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993777-eb02-4d94-b964-2d929cfff490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:30.698129 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-72993777-eb02-4d94-b964-2d929cfff490 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:30.699346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-72993777-eb02-4d94-b964-2d929cfff490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:30.781210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 249/484] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:31.069647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7579e-b55d-4e74-b1b5-911bf1eb009f None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:31.168363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:31.195525 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7579e-b55d-4e74-b1b5-911bf1eb009f None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:31.197532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72993777-eb02-4d94-b964-2d929cfff490 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:31.264248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4657a-5b45-4495-8d16-95d0fc8e69b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:31.265272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4657a-5b45-4495-8d16-95d0fc8e69b3 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:31.265721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4657a-5b45-4495-8d16-95d0fc8e69b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:31.266195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4657a-5b45-4495-8d16-95d0fc8e69b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:31.285559 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4657a-5b45-4495-8d16-95d0fc8e69b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-15c4657a-5b45-4495-8d16-95d0fc8e69b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:31.291408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:31.292319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:31.293139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:31.293139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:31.293139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:31.293139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:31.293139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:31.293139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:31.293139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:31.293139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:31.293139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:31.293139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:31.293139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:31.293139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:31.300174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 236/485] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:31.303536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4657a-5b45-4495-8d16-95d0fc8e69b3 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:31.375113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72993777-eb02-4d94-b964-2d929cfff490 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:31.377840 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:31.424431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd8e93-3454-4384-bef7-886db4ee4cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:31.425521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd8e93-3454-4384-bef7-886db4ee4cd7 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:31.425797 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd8e93-3454-4384-bef7-886db4ee4cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:31.426065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd8e93-3454-4384-bef7-886db4ee4cd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:31.439399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd8e93-3454-4384-bef7-886db4ee4cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:31.446961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4657a-5b45-4495-8d16-95d0fc8e69b3 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-f5bd8e93-3454-4384-bef7-886db4ee4cd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:31.449028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:31.450244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:31.451337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:31.451337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:31.451337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:31.451337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:31.451337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:31.451337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:31.451337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:31.451337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:31.451337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:31.451337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:31.451337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:31.451337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:31.451337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 250/486] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:31.452601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c416ee-7079-4e68-be9e-2356c8766381 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:31.453853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c416ee-7079-4e68-be9e-2356c8766381 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:31.454143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c416ee-7079-4e68-be9e-2356c8766381 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:31.455765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 251/487] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:31.456507 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c416ee-7079-4e68-be9e-2356c8766381 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:31.493359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d453fd-cc0d-4ff1-a53f-ba211b7b8514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:31.494601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d453fd-cc0d-4ff1-a53f-ba211b7b8514 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:31.495228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d453fd-cc0d-4ff1-a53f-ba211b7b8514 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:31.496045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d453fd-cc0d-4ff1-a53f-ba211b7b8514 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:31.497162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d453fd-cc0d-4ff1-a53f-ba211b7b8514 None admin] RBAC: Adding query filter params (name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:31.497654 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d453fd-cc0d-4ff1-a53f-ba211b7b8514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:31.499249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d453fd-cc0d-4ff1-a53f-ba211b7b8514 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:31.503103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:31.511841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 237/488] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:31.514586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e1f6f-eb68-450c-80d5-f6aa1b423eb8 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:31.514586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e1f6f-eb68-450c-80d5-f6aa1b423eb8 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:31.514586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e1f6f-eb68-450c-80d5-f6aa1b423eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:31.538677 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-091e1f6f-eb68-450c-80d5-f6aa1b423eb8 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:31.540979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-091e1f6f-eb68-450c-80d5-f6aa1b423eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:31.573196 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c416ee-7079-4e68-be9e-2356c8766381 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:31.608624 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d0bb2-dacc-4fc6-bf2e-4f1e50221d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:31.609220 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d0bb2-dacc-4fc6-bf2e-4f1e50221d3a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:31.609510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d0bb2-dacc-4fc6-bf2e-4f1e50221d3a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:31.609765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d0bb2-dacc-4fc6-bf2e-4f1e50221d3a None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:31.610427 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d0bb2-dacc-4fc6-bf2e-4f1e50221d3a None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:31.610766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d0bb2-dacc-4fc6-bf2e-4f1e50221d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:31.612118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d0bb2-dacc-4fc6-bf2e-4f1e50221d3a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:31.612940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 238/489] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:31.614935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4574a0e-598a-445b-8de9-633d8b4319b9 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:31.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4574a0e-598a-445b-8de9-633d8b4319b9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:31.615759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4574a0e-598a-445b-8de9-633d8b4319b9 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:31.617799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 239/490] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:31.618951 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4574a0e-598a-445b-8de9-633d8b4319b9 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:31.622536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 252/491] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:31.625653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d0bb2-dacc-4fc6-bf2e-4f1e50221d3a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:31.658958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:31.750616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d0bb2-dacc-4fc6-bf2e-4f1e50221d3a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:31.754749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4574a0e-598a-445b-8de9-633d8b4319b9 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:31.783185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:31.793145 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff5029-af1c-46dc-a030-7978ef814e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:31.793520 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e4632-25da-4b93-9541-27f203319a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:31.794194 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e4632-25da-4b93-9541-27f203319a36 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:31.794634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e4632-25da-4b93-9541-27f203319a36 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:31.794738 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff5029-af1c-46dc-a030-7978ef814e12 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:31.794811 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e4632-25da-4b93-9541-27f203319a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:31.795293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff5029-af1c-46dc-a030-7978ef814e12 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:31.795751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff5029-af1c-46dc-a030-7978ef814e12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:31.807032 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff5029-af1c-46dc-a030-7978ef814e12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:31.807182 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e4632-25da-4b93-9541-27f203319a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:31.808489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e4632-25da-4b93-9541-27f203319a36 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:31.809071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff5029-af1c-46dc-a030-7978ef814e12 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-f54e4632-25da-4b93-9541-27f203319a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:31.824761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:31.825712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:31.826603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:31.826603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:31.826603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-e0ff5029-af1c-46dc-a030-7978ef814e12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:31.828501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:31.829358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:31.830860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:31.830860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:31.830860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:31.831022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 253/492] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:37:31.831913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53ebb0-0f1d-4ee4-b742-579a0dca38d8 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:31.832190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53ebb0-0f1d-4ee4-b742-579a0dca38d8 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:31.832459 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53ebb0-0f1d-4ee4-b742-579a0dca38d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:31.833270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 240/493] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:31.834325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff5029-af1c-46dc-a030-7978ef814e12 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:31.855785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-9c53ebb0-0f1d-4ee4-b742-579a0dca38d8 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:31.857355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9c53ebb0-0f1d-4ee4-b742-579a0dca38d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:31.946594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 254/494] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:31.946594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53ebb0-0f1d-4ee4-b742-579a0dca38d8 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:31.946842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff5029-af1c-46dc-a030-7978ef814e12 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:31.982859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:31.983989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a5314-a3f3-44ec-b792-994306b5985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:31.984754 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a5314-a3f3-44ec-b792-994306b5985a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:31.985037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a5314-a3f3-44ec-b792-994306b5985a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:31.985309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a5314-a3f3-44ec-b792-994306b5985a None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:31.986561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a5314-a3f3-44ec-b792-994306b5985a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:31.986872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a5314-a3f3-44ec-b792-994306b5985a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:31.988263 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a5314-a3f3-44ec-b792-994306b5985a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:32.004072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 241/495] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:32.004932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a5314-a3f3-44ec-b792-994306b5985a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:32.059137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:32.066093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53ebb0-0f1d-4ee4-b742-579a0dca38d8 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:32.112606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2643b5-c759-4b27-a6c2-620446d09997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:32.113405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2643b5-c759-4b27-a6c2-620446d09997 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:32.113773 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2643b5-c759-4b27-a6c2-620446d09997 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:32.114138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2643b5-c759-4b27-a6c2-620446d09997 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:32.114937 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2643b5-c759-4b27-a6c2-620446d09997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:32.115334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2643b5-c759-4b27-a6c2-620446d09997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:32.117332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2643b5-c759-4b27-a6c2-620446d09997 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:32.130183 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 255/496] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:32.131201 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2643b5-c759-4b27-a6c2-620446d09997 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:32.135501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a5314-a3f3-44ec-b792-994306b5985a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:32.181373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b308dc-58c7-4077-8fe3-0224dec2a245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:32.182532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b308dc-58c7-4077-8fe3-0224dec2a245 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:32.182828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b308dc-58c7-4077-8fe3-0224dec2a245 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:32.183105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b308dc-58c7-4077-8fe3-0224dec2a245 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:32.198224 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b308dc-58c7-4077-8fe3-0224dec2a245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:32.203230 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-b4b308dc-58c7-4077-8fe3-0224dec2a245 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:32.205942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:32.207013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:32.208006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:32.208006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:32.208006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:32.208006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:32.208006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:32.208006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:32.208006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:32.208006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:32.208006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:32.208006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:32.208006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:32.208006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:32.210326 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 242/497] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:32.211865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b308dc-58c7-4077-8fe3-0224dec2a245 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:32.245059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2643b5-c759-4b27-a6c2-620446d09997 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:32.283512 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e5f7a-a536-4eaa-849f-b0f12d7b3dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:32.284242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e5f7a-a536-4eaa-849f-b0f12d7b3dd3 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:32.284513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e5f7a-a536-4eaa-849f-b0f12d7b3dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:32.284775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e5f7a-a536-4eaa-849f-b0f12d7b3dd3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:32.292742 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e5f7a-a536-4eaa-849f-b0f12d7b3dd3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:32.294170 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e5f7a-a536-4eaa-849f-b0f12d7b3dd3 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-222e5f7a-a536-4eaa-849f-b0f12d7b3dd3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:32.306160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:32.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:32.308032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:32.308032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:32.308032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:32.308032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:32.308032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:32.308032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:32.308032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:32.308032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:32.308032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:37:32.308032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:32.308032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 256/498] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:32.308610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21242f42-fe91-44d6-8462-377f1edf2ed9 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:32.308610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21242f42-fe91-44d6-8462-377f1edf2ed9 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:32.308880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21242f42-fe91-44d6-8462-377f1edf2ed9 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:32.310347 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 257/499] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:32.311045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21242f42-fe91-44d6-8462-377f1edf2ed9 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:32.319489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:32.323296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b308dc-58c7-4077-8fe3-0224dec2a245 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:32.357679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f804c7e-7b18-4cfb-b291-a00fc4305599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:32.358294 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f804c7e-7b18-4cfb-b291-a00fc4305599 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:32.358600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f804c7e-7b18-4cfb-b291-a00fc4305599 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:32.358870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f804c7e-7b18-4cfb-b291-a00fc4305599 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:32.365585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f804c7e-7b18-4cfb-b291-a00fc4305599 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:32.367103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f804c7e-7b18-4cfb-b291-a00fc4305599 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-9f804c7e-7b18-4cfb-b291-a00fc4305599 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:32.379566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:32.380528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:32.381437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:32.381437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:32.381437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:32.381437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:32.381437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:32.381437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:32.381437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:32.381437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:32.381437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:37:32.381437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:32.381437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 243/500] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 12:37:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:32.381437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f804c7e-7b18-4cfb-b291-a00fc4305599 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:32.409709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21242f42-fe91-44d6-8462-377f1edf2ed9 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:32.433682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:32.441818 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926970a6-6ad4-4005-8c5b-66b78db0d792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:32.442412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926970a6-6ad4-4005-8c5b-66b78db0d792 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:32.442715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926970a6-6ad4-4005-8c5b-66b78db0d792 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:32.446691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926970a6-6ad4-4005-8c5b-66b78db0d792 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:32.446691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926970a6-6ad4-4005-8c5b-66b78db0d792 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:32.446691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926970a6-6ad4-4005-8c5b-66b78db0d792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:32.446691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926970a6-6ad4-4005-8c5b-66b78db0d792 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:32.453919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 258/501] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:32.454672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926970a6-6ad4-4005-8c5b-66b78db0d792 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:32.486012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f804c7e-7b18-4cfb-b291-a00fc4305599 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:32.518573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ba948-3266-4860-b862-c02bb99d6dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:32.519190 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ba948-3266-4860-b862-c02bb99d6dec None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:32.519467 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ba948-3266-4860-b862-c02bb99d6dec None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:32.519733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ba948-3266-4860-b862-c02bb99d6dec None admin] PATH_INFO: `/v3/roles/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:32.530918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ba948-3266-4860-b862-c02bb99d6dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-097ba948-3266-4860-b862-c02bb99d6dec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:32.536005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:32.537102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:32.537911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:32.537911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:32.537911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:32.537911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:32.537911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:32.537911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:32.537911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:32.537911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:32.537911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:32.537911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:32.537911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:32.537911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:32.537911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 244/502] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:32.539231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70849f-3164-48dd-9e2a-045d1c51aada None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:32.539231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70849f-3164-48dd-9e2a-045d1c51aada None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:32.539417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70849f-3164-48dd-9e2a-045d1c51aada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:32.551541 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926970a6-6ad4-4005-8c5b-66b78db0d792 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:32.563851 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-bc70849f-3164-48dd-9e2a-045d1c51aada None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:32.565065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-bc70849f-3164-48dd-9e2a-045d1c51aada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:32.579151 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:32.586910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1c72a-00d8-4522-a2fc-aaaecd33bee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:32.587518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1c72a-00d8-4522-a2fc-aaaecd33bee9 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:32.587798 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1c72a-00d8-4522-a2fc-aaaecd33bee9 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:32.588071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1c72a-00d8-4522-a2fc-aaaecd33bee9 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:32.588797 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1c72a-00d8-4522-a2fc-aaaecd33bee9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:32.589098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1c72a-00d8-4522-a2fc-aaaecd33bee9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:32.590547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1c72a-00d8-4522-a2fc-aaaecd33bee9 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:32.626623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 259/503] 10.222.0.46 () {56 vars in 1137 bytes} [Fri Sep 9 12:37:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:32.627449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1c72a-00d8-4522-a2fc-aaaecd33bee9 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:32.631524 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 245/504] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:32.632489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70849f-3164-48dd-9e2a-045d1c51aada None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:32.675102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:32.731018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1c72a-00d8-4522-a2fc-aaaecd33bee9 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:32.751413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70849f-3164-48dd-9e2a-045d1c51aada None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:32.763878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23191c-f1f1-4dc1-a094-2c3fdf296238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:32.764472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23191c-f1f1-4dc1-a094-2c3fdf296238 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:32.764744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23191c-f1f1-4dc1-a094-2c3fdf296238 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:32.765033 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23191c-f1f1-4dc1-a094-2c3fdf296238 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:32.765729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23191c-f1f1-4dc1-a094-2c3fdf296238 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:32.766028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23191c-f1f1-4dc1-a094-2c3fdf296238 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:32.767645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23191c-f1f1-4dc1-a094-2c3fdf296238 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:32.785160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 260/505] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 12:37:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:32.785988 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23191c-f1f1-4dc1-a094-2c3fdf296238 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:32.794924 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da51f6-686d-4521-86c1-89fe37111482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:32.795758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da51f6-686d-4521-86c1-89fe37111482 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:32.796101 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da51f6-686d-4521-86c1-89fe37111482 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:32.796437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da51f6-686d-4521-86c1-89fe37111482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:32.807548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da51f6-686d-4521-86c1-89fe37111482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:32.808979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da51f6-686d-4521-86c1-89fe37111482 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-47da51f6-686d-4521-86c1-89fe37111482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:32.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:32.824639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:32.825490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:32.825490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:32.825490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:32.825490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 246/506] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:32.825716 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da51f6-686d-4521-86c1-89fe37111482 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:32.829230 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:32.909770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23191c-f1f1-4dc1-a094-2c3fdf296238 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:32.912238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:32.932937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da51f6-686d-4521-86c1-89fe37111482 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:32.941754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844915e9-1d3c-483c-8d6c-9365d518ff89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:32.942366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844915e9-1d3c-483c-8d6c-9365d518ff89 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:32.942853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844915e9-1d3c-483c-8d6c-9365d518ff89 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:32.943296 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844915e9-1d3c-483c-8d6c-9365d518ff89 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:32.943949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844915e9-1d3c-483c-8d6c-9365d518ff89 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:32.944569 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844915e9-1d3c-483c-8d6c-9365d518ff89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:32.946454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844915e9-1d3c-483c-8d6c-9365d518ff89 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:32.953993 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 261/507] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:32.954792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844915e9-1d3c-483c-8d6c-9365d518ff89 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:32.966620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8223793-a1c6-4086-99b8-9e38f24a8d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:32.967221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8223793-a1c6-4086-99b8-9e38f24a8d5c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:32.967495 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8223793-a1c6-4086-99b8-9e38f24a8d5c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:32.967769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8223793-a1c6-4086-99b8-9e38f24a8d5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:33.035127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8223793-a1c6-4086-99b8-9e38f24a8d5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:33.036483 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8223793-a1c6-4086-99b8-9e38f24a8d5c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:33.038241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:33.051589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844915e9-1d3c-483c-8d6c-9365d518ff89 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:33.084709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4b53c-2bee-4143-87c4-9588d77480e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:33.085299 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4b53c-2bee-4143-87c4-9588d77480e7 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:33.086170 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4b53c-2bee-4143-87c4-9588d77480e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:33.086441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4b53c-2bee-4143-87c4-9588d77480e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:33.120719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:33.133680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 247/508] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:33.134558 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8223793-a1c6-4086-99b8-9e38f24a8d5c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:33.155747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4b53c-2bee-4143-87c4-9588d77480e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:33.157216 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4b53c-2bee-4143-87c4-9588d77480e7 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:33.235046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8223793-a1c6-4086-99b8-9e38f24a8d5c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:33.237323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:33.262296 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 262/509] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:33.263164 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4b53c-2bee-4143-87c4-9588d77480e7 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:33.270045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2a660-3145-428b-9840-cd1f927ad22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:33.270683 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2a660-3145-428b-9840-cd1f927ad22c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:33.270960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2a660-3145-428b-9840-cd1f927ad22c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:33.271226 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2a660-3145-428b-9840-cd1f927ad22c None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:33.271849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2a660-3145-428b-9840-cd1f927ad22c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:33.272144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2a660-3145-428b-9840-cd1f927ad22c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:33.273542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2a660-3145-428b-9840-cd1f927ad22c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:33.284696 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 248/510] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:33.285534 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2a660-3145-428b-9840-cd1f927ad22c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:33.333145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:33.365882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4b53c-2bee-4143-87c4-9588d77480e7 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:33.384063 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2a660-3145-428b-9840-cd1f927ad22c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:33.399776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd6e54-f1af-4e70-b2e1-60c9efd7fa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:33.400396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd6e54-f1af-4e70-b2e1-60c9efd7fa18 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:33.400672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd6e54-f1af-4e70-b2e1-60c9efd7fa18 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:33.400945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd6e54-f1af-4e70-b2e1-60c9efd7fa18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:33.411183 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd6e54-f1af-4e70-b2e1-60c9efd7fa18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:33.412617 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd6e54-f1af-4e70-b2e1-60c9efd7fa18 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:33.418022 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7c33f-2de8-4bc3-891b-997d1de8376f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:33.418670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7c33f-2de8-4bc3-891b-997d1de8376f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:33.418953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7c33f-2de8-4bc3-891b-997d1de8376f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:33.419224 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7c33f-2de8-4bc3-891b-997d1de8376f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-8acd6e54-f1af-4e70-b2e1-60c9efd7fa18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:33.427479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:33.428377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:33.429719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:33.429719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:33.429719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:33.429719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 263/511] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:33.429719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd6e54-f1af-4e70-b2e1-60c9efd7fa18 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:33.432188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7c33f-2de8-4bc3-891b-997d1de8376f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:33.433743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7c33f-2de8-4bc3-891b-997d1de8376f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-73e7c33f-2de8-4bc3-891b-997d1de8376f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:33.448549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:33.449648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:33.450841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:33.450841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:33.450841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:33.450841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 249/512] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:33.450841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7c33f-2de8-4bc3-891b-997d1de8376f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:33.469805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:33.538082 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd6e54-f1af-4e70-b2e1-60c9efd7fa18 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:33.553284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7c33f-2de8-4bc3-891b-997d1de8376f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:33.555300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:33.571754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68d5ff-4614-4f59-b6f8-6b58ec0f4aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:33.572377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68d5ff-4614-4f59-b6f8-6b58ec0f4aec None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:33.572990 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68d5ff-4614-4f59-b6f8-6b58ec0f4aec None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:33.572990 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68d5ff-4614-4f59-b6f8-6b58ec0f4aec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:33.584236 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68d5ff-4614-4f59-b6f8-6b58ec0f4aec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:33.586248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68d5ff-4614-4f59-b6f8-6b58ec0f4aec None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:33.586782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36721154-a977-49a0-a593-05da731455df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:33.587409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36721154-a977-49a0-a593-05da731455df None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:33.587690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36721154-a977-49a0-a593-05da731455df None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:33.587966 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36721154-a977-49a0-a593-05da731455df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:33.598087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36721154-a977-49a0-a593-05da731455df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-0e68d5ff-4614-4f59-b6f8-6b58ec0f4aec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:33.600193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:33.602223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:33.603281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:33.603281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:33.603281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:33.603486 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36721154-a977-49a0-a593-05da731455df None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:33.603565 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 264/513] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:33.603565 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68d5ff-4614-4f59-b6f8-6b58ec0f4aec None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-36721154-a977-49a0-a593-05da731455df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:33.618077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:33.619229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:33.620253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:33.620253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:33.620253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:33.620253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 250/514] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:33.620540 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36721154-a977-49a0-a593-05da731455df None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:33.683976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:33.707001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68d5ff-4614-4f59-b6f8-6b58ec0f4aec None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:33.722109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36721154-a977-49a0-a593-05da731455df None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:33.739976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34b3b9-dad2-4a95-a102-d588dbb15d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:33.740598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b3b9-dad2-4a95-a102-d588dbb15d14 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:33.740874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b3b9-dad2-4a95-a102-d588dbb15d14 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:33.741558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b3b9-dad2-4a95-a102-d588dbb15d14 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:33.748930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34b3b9-dad2-4a95-a102-d588dbb15d14 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:33.750350 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34b3b9-dad2-4a95-a102-d588dbb15d14 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:33.754903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337d442-1e4f-4005-9d62-58bf9c5fdf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:33.755501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337d442-1e4f-4005-9d62-58bf9c5fdf33 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:33.755783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337d442-1e4f-4005-9d62-58bf9c5fdf33 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:33.756055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337d442-1e4f-4005-9d62-58bf9c5fdf33 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:33.756687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337d442-1e4f-4005-9d62-58bf9c5fdf33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:33.756989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337d442-1e4f-4005-9d62-58bf9c5fdf33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:33.759003 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337d442-1e4f-4005-9d62-58bf9c5fdf33 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-ce34b3b9-dad2-4a95-a102-d588dbb15d14 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:33.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:33.762673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:33.763579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:33.763579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:33.763579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:33.763579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:33.763579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:33.763579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:33.763579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:33.763579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:33.763579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 12:37:33.763579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:33.763579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 265/515] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:33.763579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34b3b9-dad2-4a95-a102-d588dbb15d14 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:33.773625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 251/516] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:33.774440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337d442-1e4f-4005-9d62-58bf9c5fdf33 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:33.793599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:33.861656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34b3b9-dad2-4a95-a102-d588dbb15d14 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:33.872402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337d442-1e4f-4005-9d62-58bf9c5fdf33 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:33.893699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073799a7-9e94-4e92-84d5-63ba7352a702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:33.894294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073799a7-9e94-4e92-84d5-63ba7352a702 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:33.894607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073799a7-9e94-4e92-84d5-63ba7352a702 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:33.894882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073799a7-9e94-4e92-84d5-63ba7352a702 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:33.895511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073799a7-9e94-4e92-84d5-63ba7352a702 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:33.895807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073799a7-9e94-4e92-84d5-63ba7352a702 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:33.896949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:33.897154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073799a7-9e94-4e92-84d5-63ba7352a702 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:33.905600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071d872-0d19-4f33-99cf-a44098cbc544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:33.906249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071d872-0d19-4f33-99cf-a44098cbc544 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:33.906558 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071d872-0d19-4f33-99cf-a44098cbc544 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:33.906833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071d872-0d19-4f33-99cf-a44098cbc544 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:33.913276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071d872-0d19-4f33-99cf-a44098cbc544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:33.913276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071d872-0d19-4f33-99cf-a44098cbc544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:33.913476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 266/517] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:33.914108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071d872-0d19-4f33-99cf-a44098cbc544 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:33.914216 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073799a7-9e94-4e92-84d5-63ba7352a702 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:33.920980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 252/518] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:33.921785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071d872-0d19-4f33-99cf-a44098cbc544 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:33.997370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:34.007600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073799a7-9e94-4e92-84d5-63ba7352a702 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:34.025225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071d872-0d19-4f33-99cf-a44098cbc544 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:34.042170 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf073bc-22cb-4799-9a2b-8b8cbb9fb223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:34.042810 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf073bc-22cb-4799-9a2b-8b8cbb9fb223 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:34.043088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf073bc-22cb-4799-9a2b-8b8cbb9fb223 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:34.043358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf073bc-22cb-4799-9a2b-8b8cbb9fb223 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:34.044047 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf073bc-22cb-4799-9a2b-8b8cbb9fb223 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:34.044298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf073bc-22cb-4799-9a2b-8b8cbb9fb223 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:34.045649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf073bc-22cb-4799-9a2b-8b8cbb9fb223 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:34.057452 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 267/519] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:34.058364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf073bc-22cb-4799-9a2b-8b8cbb9fb223 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:34.058898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe448f-d8b7-4b0c-b7e2-df9526858c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:34.059574 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe448f-d8b7-4b0c-b7e2-df9526858c95 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:34.060181 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe448f-d8b7-4b0c-b7e2-df9526858c95 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:34.060522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe448f-d8b7-4b0c-b7e2-df9526858c95 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:34.061320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe448f-d8b7-4b0c-b7e2-df9526858c95 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:34.061725 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe448f-d8b7-4b0c-b7e2-df9526858c95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:34.063518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe448f-d8b7-4b0c-b7e2-df9526858c95 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:34.099201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 253/520] 10.222.0.46 () {56 vars in 1137 bytes} [Fri Sep 9 12:37:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:34.103388 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe448f-d8b7-4b0c-b7e2-df9526858c95 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:34.149915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:34.162067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf073bc-22cb-4799-9a2b-8b8cbb9fb223 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:34.194588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58018f2b-f037-4858-910d-4bc51c76892e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:34.195205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58018f2b-f037-4858-910d-4bc51c76892e None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:34.195477 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58018f2b-f037-4858-910d-4bc51c76892e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:34.195743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58018f2b-f037-4858-910d-4bc51c76892e None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:34.196374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58018f2b-f037-4858-910d-4bc51c76892e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:34.197790 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58018f2b-f037-4858-910d-4bc51c76892e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:34.202906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe448f-d8b7-4b0c-b7e2-df9526858c95 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:34.236089 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a5de7-4dbc-4d08-892d-5fd0f03ed05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:34.236701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a5de7-4dbc-4d08-892d-5fd0f03ed05c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:34.236977 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a5de7-4dbc-4d08-892d-5fd0f03ed05c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:34.237245 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a5de7-4dbc-4d08-892d-5fd0f03ed05c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:34.243925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a5de7-4dbc-4d08-892d-5fd0f03ed05c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:34.245349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a5de7-4dbc-4d08-892d-5fd0f03ed05c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-b68a5de7-4dbc-4d08-892d-5fd0f03ed05c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:34.256432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:34.257623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:34.258552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:34.258552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:34.258552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:34.258552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:34.258552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:34.258552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:34.258552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:34.258552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:34.258552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 12:37:34.258552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:34.258552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 254/521] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:34.258552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a5de7-4dbc-4d08-892d-5fd0f03ed05c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:34.274514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 268/522] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:37:34 2022] POST /identity/v3/users => generated 247 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:37:34.275351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58018f2b-f037-4858-910d-4bc51c76892e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:34.287215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:34.381375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a5de7-4dbc-4d08-892d-5fd0f03ed05c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:34.384357 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:34.406217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58018f2b-f037-4858-910d-4bc51c76892e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:34.421160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0050ff5-d0b2-4216-98d2-f720d595dbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:34.421812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0050ff5-d0b2-4216-98d2-f720d595dbb8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:34.422231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0050ff5-d0b2-4216-98d2-f720d595dbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:34.422818 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0050ff5-d0b2-4216-98d2-f720d595dbb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:34.432343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0050ff5-d0b2-4216-98d2-f720d595dbb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:34.433819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0050ff5-d0b2-4216-98d2-f720d595dbb8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:34.444700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6bbb87-ac83-4221-ad23-37b757c98d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:34.445671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bbb87-ac83-4221-ad23-37b757c98d10 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:34.446146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bbb87-ac83-4221-ad23-37b757c98d10 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:34.446606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bbb87-ac83-4221-ad23-37b757c98d10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-b0050ff5-d0b2-4216-98d2-f720d595dbb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:34.447744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:34.448664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:34.449529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:34.449529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:34.449529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:34.449529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:34.449529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:34.449529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:34.449529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:34.449529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:34.449529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 255/523] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:34.450044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0050ff5-d0b2-4216-98d2-f720d595dbb8 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:34.457004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bbb87-ac83-4221-ad23-37b757c98d10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:34.458447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bbb87-ac83-4221-ad23-37b757c98d10 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-ad6bbb87-ac83-4221-ad23-37b757c98d10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:34.472903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:34.473781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:34.474687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:34.474687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:34.474687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:34.474687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:34.474687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:34.474687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:34.474687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:34.474687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:34.477174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 269/524] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:34.478304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6bbb87-ac83-4221-ad23-37b757c98d10 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:34.550855 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:34.573494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0050ff5-d0b2-4216-98d2-f720d595dbb8 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:34.588962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6bbb87-ac83-4221-ad23-37b757c98d10 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:34.610004 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a56fbb-3209-4dc8-9194-e7bd3dd69e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:34.610680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a56fbb-3209-4dc8-9194-e7bd3dd69e95 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:34.610962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a56fbb-3209-4dc8-9194-e7bd3dd69e95 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:34.611283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a56fbb-3209-4dc8-9194-e7bd3dd69e95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:34.625915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7dbbf-fba0-4c71-b416-313fd2dba073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:34.626836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7dbbf-fba0-4c71-b416-313fd2dba073 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:34.626836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7dbbf-fba0-4c71-b416-313fd2dba073 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:34.627103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7dbbf-fba0-4c71-b416-313fd2dba073 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:34.627858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7dbbf-fba0-4c71-b416-313fd2dba073 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:34.628088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7dbbf-fba0-4c71-b416-313fd2dba073 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:34.629498 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7dbbf-fba0-4c71-b416-313fd2dba073 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:34.643172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:34.647638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 270/525] 10.222.0.46 () {56 vars in 1139 bytes} [Fri Sep 9 12:37:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:34.649312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7dbbf-fba0-4c71-b416-313fd2dba073 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:34.686792 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a56fbb-3209-4dc8-9194-e7bd3dd69e95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:34.688258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a56fbb-3209-4dc8-9194-e7bd3dd69e95 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:34.758038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7dbbf-fba0-4c71-b416-313fd2dba073 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:34.783785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:34.796343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 256/526] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:34.796343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a56fbb-3209-4dc8-9194-e7bd3dd69e95 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:34.799427 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dd5b0-d76c-457b-a3aa-08390667cb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:34.802626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dd5b0-d76c-457b-a3aa-08390667cb30 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:34.802626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dd5b0-d76c-457b-a3aa-08390667cb30 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:34.802626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dd5b0-d76c-457b-a3aa-08390667cb30 None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:34.802626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47dd5b0-d76c-457b-a3aa-08390667cb30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:34.802626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47dd5b0-d76c-457b-a3aa-08390667cb30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:34.802964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47dd5b0-d76c-457b-a3aa-08390667cb30 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:34.825662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 271/527] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:34.825662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dd5b0-d76c-457b-a3aa-08390667cb30 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:34.867959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:34.907743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a56fbb-3209-4dc8-9194-e7bd3dd69e95 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:34.925515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dd5b0-d76c-457b-a3aa-08390667cb30 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:34.943988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfdce8-51c5-4c22-b72f-bfe7f8954952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:34.944595 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfdce8-51c5-4c22-b72f-bfe7f8954952 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:34.945056 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfdce8-51c5-4c22-b72f-bfe7f8954952 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:34.945433 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfdce8-51c5-4c22-b72f-bfe7f8954952 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:34.946184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfdce8-51c5-4c22-b72f-bfe7f8954952 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:34.946503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfdce8-51c5-4c22-b72f-bfe7f8954952 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:34.947962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfdce8-51c5-4c22-b72f-bfe7f8954952 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:34.959717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2a943-ff85-438e-98e1-6985a0c1a3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:34.960323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2a943-ff85-438e-98e1-6985a0c1a3e6 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:34.961257 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2a943-ff85-438e-98e1-6985a0c1a3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:34.961533 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2a943-ff85-438e-98e1-6985a0c1a3e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:34.971826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 257/528] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:34.971826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfdce8-51c5-4c22-b72f-bfe7f8954952 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:35.017848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:35.034416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2a943-ff85-438e-98e1-6985a0c1a3e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:35.035879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2a943-ff85-438e-98e1-6985a0c1a3e6 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:35.079927 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfdce8-51c5-4c22-b72f-bfe7f8954952 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:35.108775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:35.123255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a543e2-f506-48ff-81ce-c312ba31d497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:35.124127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a543e2-f506-48ff-81ce-c312ba31d497 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:35.124242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a543e2-f506-48ff-81ce-c312ba31d497 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:35.124491 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a543e2-f506-48ff-81ce-c312ba31d497 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:35.135945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a543e2-f506-48ff-81ce-c312ba31d497 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:35.137807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a543e2-f506-48ff-81ce-c312ba31d497 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:35.148714 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 272/529] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:35.150026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2a943-ff85-438e-98e1-6985a0c1a3e6 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-c7a543e2-f506-48ff-81ce-c312ba31d497 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:35.155171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:35.156292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:35.157188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:35.157188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:35.157188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:35.157188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 258/530] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:35.157188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a543e2-f506-48ff-81ce-c312ba31d497 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:35.247350 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:35.263649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2a943-ff85-438e-98e1-6985a0c1a3e6 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:35.271074 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a543e2-f506-48ff-81ce-c312ba31d497 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:35.301656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27e9b3-6221-4202-a3a1-b437d7e0d1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:35.302559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27e9b3-6221-4202-a3a1-b437d7e0d1b9 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:35.302928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27e9b3-6221-4202-a3a1-b437d7e0d1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:35.303209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27e9b3-6221-4202-a3a1-b437d7e0d1b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:35.304145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27e9b3-6221-4202-a3a1-b437d7e0d1b9 None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=002c117a85c1452e9fae80c755c6622f) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:35.304477 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27e9b3-6221-4202-a3a1-b437d7e0d1b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:35.306189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a7904-41d1-469a-b1fa-422f992b7638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:35.306304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27e9b3-6221-4202-a3a1-b437d7e0d1b9 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:35.306823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a7904-41d1-469a-b1fa-422f992b7638 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:35.307104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a7904-41d1-469a-b1fa-422f992b7638 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:35.307377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a7904-41d1-469a-b1fa-422f992b7638 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:35.308093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a7904-41d1-469a-b1fa-422f992b7638 None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=f29af98c5fba4569b16edf240b13c5c5) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:35.308398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a7904-41d1-469a-b1fa-422f992b7638 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:35.310069 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a7904-41d1-469a-b1fa-422f992b7638 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:35.322949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 273/531] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:35 2022] GET /identity/v3/role_assignments?user.id=002c117a85c1452e9fae80c755c6622f&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:35.322949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27e9b3-6221-4202-a3a1-b437d7e0d1b9 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:35.323248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 259/532] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:35 2022] GET /identity/v3/role_assignments?user.id=f29af98c5fba4569b16edf240b13c5c5&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:35.323248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a7904-41d1-469a-b1fa-422f992b7638 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:35.418232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:35.528389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a7904-41d1-469a-b1fa-422f992b7638 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:35.539140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27e9b3-6221-4202-a3a1-b437d7e0d1b9 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:35.588650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ceaf6-f9d1-4f49-98b1-99efa344dff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:35.589668 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ceaf6-f9d1-4f49-98b1-99efa344dff3 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:35.590136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ceaf6-f9d1-4f49-98b1-99efa344dff3 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:35.590680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ceaf6-f9d1-4f49-98b1-99efa344dff3 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:35.591837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ceaf6-f9d1-4f49-98b1-99efa344dff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:35.592337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ceaf6-f9d1-4f49-98b1-99efa344dff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:35.594719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ceaf6-f9d1-4f49-98b1-99efa344dff3 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:35.614361 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb91ece-d697-4dba-9430-5c9eee087793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:35.615041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb91ece-d697-4dba-9430-5c9eee087793 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:35.615320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb91ece-d697-4dba-9430-5c9eee087793 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:35.615585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb91ece-d697-4dba-9430-5c9eee087793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:35.628155 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 260/533] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:35.635544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ceaf6-f9d1-4f49-98b1-99efa344dff3 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:35.639894 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb91ece-d697-4dba-9430-5c9eee087793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:35.641337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb91ece-d697-4dba-9430-5c9eee087793 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:35.659306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-abb91ece-d697-4dba-9430-5c9eee087793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:35.668598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:35.669537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:35.670380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:35.670380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:35.670380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:35.675392 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 274/534] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:35.677615 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb91ece-d697-4dba-9430-5c9eee087793 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:35.822959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:35.825276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ceaf6-f9d1-4f49-98b1-99efa344dff3 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:35.873296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c0ec0-b054-47ce-a8e9-b3b882f1a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:35.873945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c0ec0-b054-47ce-a8e9-b3b882f1a604 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:35.874227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c0ec0-b054-47ce-a8e9-b3b882f1a604 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:35.874520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c0ec0-b054-47ce-a8e9-b3b882f1a604 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:35.881898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c0ec0-b054-47ce-a8e9-b3b882f1a604 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:35.883923 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c0ec0-b054-47ce-a8e9-b3b882f1a604 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:35.886421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb91ece-d697-4dba-9430-5c9eee087793 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-7c3c0ec0-b054-47ce-a8e9-b3b882f1a604 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:35.896939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:35.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:35.899014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:35.899014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:35.899014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:35.899014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:35.899014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:35.899014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:35.899014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:35.899014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:35.899984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 261/535] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:35.905517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c0ec0-b054-47ce-a8e9-b3b882f1a604 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:35.956062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af288ca-cb94-413e-abfa-514f5ddab32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:35.957024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af288ca-cb94-413e-abfa-514f5ddab32a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:35.957524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af288ca-cb94-413e-abfa-514f5ddab32a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:35.957989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af288ca-cb94-413e-abfa-514f5ddab32a None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:35.959164 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af288ca-cb94-413e-abfa-514f5ddab32a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:35.959893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af288ca-cb94-413e-abfa-514f5ddab32a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:35.962544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af288ca-cb94-413e-abfa-514f5ddab32a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:35.992270 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 275/536] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:35.998208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af288ca-cb94-413e-abfa-514f5ddab32a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:36.016545 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c0ec0-b054-47ce-a8e9-b3b882f1a604 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:36.051765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfe54f-dadf-418c-88d2-cf6670f3bb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:36.052383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfe54f-dadf-418c-88d2-cf6670f3bb64 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:36.052667 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfe54f-dadf-418c-88d2-cf6670f3bb64 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:36.052944 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfe54f-dadf-418c-88d2-cf6670f3bb64 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:36.053632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfe54f-dadf-418c-88d2-cf6670f3bb64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:36.053961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfe54f-dadf-418c-88d2-cf6670f3bb64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:36.055942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfe54f-dadf-418c-88d2-cf6670f3bb64 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:36.069058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 262/537] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:36.075619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfe54f-dadf-418c-88d2-cf6670f3bb64 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:36.090958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:36.100996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:36.162678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af288ca-cb94-413e-abfa-514f5ddab32a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:36.222993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfe54f-dadf-418c-88d2-cf6670f3bb64 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:36.224568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368a53d-f141-4b2d-8d76-79c854675231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:36.225312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368a53d-f141-4b2d-8d76-79c854675231 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:36.225590 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368a53d-f141-4b2d-8d76-79c854675231 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:36.225859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368a53d-f141-4b2d-8d76-79c854675231 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:36.239257 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368a53d-f141-4b2d-8d76-79c854675231 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:36.241071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368a53d-f141-4b2d-8d76-79c854675231 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-4368a53d-f141-4b2d-8d76-79c854675231 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:36.267727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:36.268650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:36.269506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:36.269506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:36.269506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:36.269506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:36.269506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:36.269506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:36.269506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:36.269506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:36.276317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 276/538] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:37:36.284209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368a53d-f141-4b2d-8d76-79c854675231 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:36.301301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdcb06-0b18-42b0-b9e6-3fd6efa1041d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:36.302714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdcb06-0b18-42b0-b9e6-3fd6efa1041d None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:36.303194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdcb06-0b18-42b0-b9e6-3fd6efa1041d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:36.303673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdcb06-0b18-42b0-b9e6-3fd6efa1041d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:36.305664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bdcb06-0b18-42b0-b9e6-3fd6efa1041d None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=eff8fcb7fa2740bbb801badf800d2d6a) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:36.306727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bdcb06-0b18-42b0-b9e6-3fd6efa1041d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:36.310747 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bdcb06-0b18-42b0-b9e6-3fd6efa1041d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:36.320785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 263/539] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:36 2022] GET /identity/v3/role_assignments?user.id=eff8fcb7fa2740bbb801badf800d2d6a&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:36.422462 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368a53d-f141-4b2d-8d76-79c854675231 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:36.445458 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:36.466399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42853a-e47d-42c0-9b81-ff5bb8807fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:36.471507 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42853a-e47d-42c0-9b81-ff5bb8807fed None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:36.471507 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42853a-e47d-42c0-9b81-ff5bb8807fed None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:36.471507 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42853a-e47d-42c0-9b81-ff5bb8807fed None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:36.471507 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42853a-e47d-42c0-9b81-ff5bb8807fed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:36.471507 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42853a-e47d-42c0-9b81-ff5bb8807fed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:36.471507 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42853a-e47d-42c0-9b81-ff5bb8807fed None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:36.495686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 277/540] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:36.501309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdcb06-0b18-42b0-b9e6-3fd6efa1041d None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:36.622153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:36.639417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdcb06-0b18-42b0-b9e6-3fd6efa1041d None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:36.643800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db882a0-46a0-40af-9d45-be6a87a1e37b None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:36.644287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db882a0-46a0-40af-9d45-be6a87a1e37b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:36.644372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db882a0-46a0-40af-9d45-be6a87a1e37b None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:36.646590 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 278/541] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:36.661843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f439d82-11ef-41fe-8efb-feadfd980523 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:36.662143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f439d82-11ef-41fe-8efb-feadfd980523 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:36.662388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f439d82-11ef-41fe-8efb-feadfd980523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:36.676251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661d250-dd8d-4289-9f1f-3dcbbd69ddac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:36.676791 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661d250-dd8d-4289-9f1f-3dcbbd69ddac None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:36.677076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661d250-dd8d-4289-9f1f-3dcbbd69ddac None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:36.677368 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661d250-dd8d-4289-9f1f-3dcbbd69ddac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:36.678086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661d250-dd8d-4289-9f1f-3dcbbd69ddac None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=e46bd9d49bf64013ae58770b0284d81e) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:36.678395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661d250-dd8d-4289-9f1f-3dcbbd69ddac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:36.680347 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661d250-dd8d-4289-9f1f-3dcbbd69ddac None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:36.686701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 264/542] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:36 2022] GET /identity/v3/role_assignments?user.id=e46bd9d49bf64013ae58770b0284d81e&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:36.696088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-1f439d82-11ef-41fe-8efb-feadfd980523 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:36.697490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1f439d82-11ef-41fe-8efb-feadfd980523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:36.775076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 279/543] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:37:37.269864 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661d250-dd8d-4289-9f1f-3dcbbd69ddac None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:37.385964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661d250-dd8d-4289-9f1f-3dcbbd69ddac None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:37.421346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a9964-f982-4ab3-b4cb-d4c4bf9a4852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:37.421988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a9964-f982-4ab3-b4cb-d4c4bf9a4852 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:37.422270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a9964-f982-4ab3-b4cb-d4c4bf9a4852 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:37.422573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a9964-f982-4ab3-b4cb-d4c4bf9a4852 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:37.433276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a9964-f982-4ab3-b4cb-d4c4bf9a4852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-983a9964-f982-4ab3-b4cb-d4c4bf9a4852 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:37.438823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:37.439703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:37.440499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:37.440499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:37.440499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:37.440499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:37.440499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:37.440499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:37.440499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:37.440499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:37.440499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:37.440499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:37.440499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:37.440499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:37.440499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 265/544] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:37.448160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f439d82-11ef-41fe-8efb-feadfd980523 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:37.598109 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f439d82-11ef-41fe-8efb-feadfd980523 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:37.614248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:37.646084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41c4ff-231c-4596-8bff-356ef6339972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:37.646709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41c4ff-231c-4596-8bff-356ef6339972 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:37.646918 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41c4ff-231c-4596-8bff-356ef6339972 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:37.647178 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41c4ff-231c-4596-8bff-356ef6339972 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:37.647903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41c4ff-231c-4596-8bff-356ef6339972 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:37.648713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41c4ff-231c-4596-8bff-356ef6339972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:37.650583 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41c4ff-231c-4596-8bff-356ef6339972 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:37.661499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 280/545] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:37.669449 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a9964-f982-4ab3-b4cb-d4c4bf9a4852 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:37.684498 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:37.839315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a9964-f982-4ab3-b4cb-d4c4bf9a4852 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:37.892976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772a1be-ff74-4341-85c8-85afe723fc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:37.893965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772a1be-ff74-4341-85c8-85afe723fc84 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:37.894454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772a1be-ff74-4341-85c8-85afe723fc84 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:37.894938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772a1be-ff74-4341-85c8-85afe723fc84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:37.914794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772a1be-ff74-4341-85c8-85afe723fc84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:37.916595 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772a1be-ff74-4341-85c8-85afe723fc84 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-9772a1be-ff74-4341-85c8-85afe723fc84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:37.938249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:37.939325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:37.940189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:37.940189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:37.940189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:37.941971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 266/546] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:37.948299 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41c4ff-231c-4596-8bff-356ef6339972 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:37.990067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fd5cc-1d8a-4a7b-a5f1-aa6bb8c6af7a None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:37.990855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fd5cc-1d8a-4a7b-a5f1-aa6bb8c6af7a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:37.991871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fd5cc-1d8a-4a7b-a5f1-aa6bb8c6af7a None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:37.993969 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 267/547] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:38.000872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d70ca-6414-4e0b-8e17-343f0d834b56 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:38.001507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d70ca-6414-4e0b-8e17-343f0d834b56 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:38.001982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d70ca-6414-4e0b-8e17-343f0d834b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:38.035584 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-b61d70ca-6414-4e0b-8e17-343f0d834b56 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:38.037239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b61d70ca-6414-4e0b-8e17-343f0d834b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:38.072178 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41c4ff-231c-4596-8bff-356ef6339972 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:38.123425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bce73c-064a-4c6b-acea-4bcf42f10d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:38.124054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bce73c-064a-4c6b-acea-4bcf42f10d05 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:38.124330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bce73c-064a-4c6b-acea-4bcf42f10d05 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:38.124600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bce73c-064a-4c6b-acea-4bcf42f10d05 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:38.125258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bce73c-064a-4c6b-acea-4bcf42f10d05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:38.125557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bce73c-064a-4c6b-acea-4bcf42f10d05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:38.127003 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bce73c-064a-4c6b-acea-4bcf42f10d05 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:38.136918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:38.140102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 281/548] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:38.141395 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec6c72-7186-41c9-a2a3-f8610e326b73 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:38.141683 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec6c72-7186-41c9-a2a3-f8610e326b73 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:38.141964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec6c72-7186-41c9-a2a3-f8610e326b73 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:38.147609 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 282/549] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:38.152270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 268/550] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:38.152270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d70ca-6414-4e0b-8e17-343f0d834b56 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:38.158866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5414584-d3a2-445f-9d5b-acbb0f943df3 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:38.158866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5414584-d3a2-445f-9d5b-acbb0f943df3 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:38.158866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5414584-d3a2-445f-9d5b-acbb0f943df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:38.190943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b5414584-d3a2-445f-9d5b-acbb0f943df3 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:38.194246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b5414584-d3a2-445f-9d5b-acbb0f943df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:38.246850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:38.298983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d70ca-6414-4e0b-8e17-343f0d834b56 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:38.299996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 283/551] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:38.335262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a74414-6e94-4ca9-8536-7c7886ce1eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:38.336469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a74414-6e94-4ca9-8536-7c7886ce1eb6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:38.336958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a74414-6e94-4ca9-8536-7c7886ce1eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:38.337455 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a74414-6e94-4ca9-8536-7c7886ce1eb6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:38.354752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a74414-6e94-4ca9-8536-7c7886ce1eb6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:38.356625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a74414-6e94-4ca9-8536-7c7886ce1eb6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-07a74414-6e94-4ca9-8536-7c7886ce1eb6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:38.370460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:38.371446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:38.372369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:38.372369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:38.372369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:38.372369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:38.372369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:38.372369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:38.372369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:38.372369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:38.372369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:37:38.372369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:38.375027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 269/552] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:37:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:38.383554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5414584-d3a2-445f-9d5b-acbb0f943df3 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:38.500514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a74414-6e94-4ca9-8536-7c7886ce1eb6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:38.524707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5414584-d3a2-445f-9d5b-acbb0f943df3 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:38.592203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeff96f-9dcb-4244-9bb5-3acaced7d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:38.592816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeff96f-9dcb-4244-9bb5-3acaced7d15d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:38.593091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeff96f-9dcb-4244-9bb5-3acaced7d15d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:38.593359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeff96f-9dcb-4244-9bb5-3acaced7d15d None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:38.594091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeff96f-9dcb-4244-9bb5-3acaced7d15d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:38.594396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeff96f-9dcb-4244-9bb5-3acaced7d15d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:38.595894 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeff96f-9dcb-4244-9bb5-3acaced7d15d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:38.601804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:38.632236 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 284/553] 10.222.0.46 () {56 vars in 1135 bytes} [Fri Sep 9 12:37:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:38.634837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeff96f-9dcb-4244-9bb5-3acaced7d15d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:38.668737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:38.687297 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a74414-6e94-4ca9-8536-7c7886ce1eb6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:38.735881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ccb7f-1036-4f72-81e0-9096c0566f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:38.736508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ccb7f-1036-4f72-81e0-9096c0566f88 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:38.736792 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ccb7f-1036-4f72-81e0-9096c0566f88 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:38.737073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ccb7f-1036-4f72-81e0-9096c0566f88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:38.752314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ccb7f-1036-4f72-81e0-9096c0566f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-190ccb7f-1036-4f72-81e0-9096c0566f88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:38.759562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:38.760529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:38.761446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:38.761446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:38.761446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:38.761446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:38.761446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:38.761446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:38.761446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:38.761446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:38.761446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:38.761446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:38.761446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:38.761446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:38.762144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 270/554] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:38.767875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ccb7f-1036-4f72-81e0-9096c0566f88 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:38.780432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeff96f-9dcb-4244-9bb5-3acaced7d15d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:38.824952 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec031765-46b3-4842-94c8-ebd4d18ace94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:38.826337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec031765-46b3-4842-94c8-ebd4d18ace94 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:38.827153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec031765-46b3-4842-94c8-ebd4d18ace94 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:38.827637 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec031765-46b3-4842-94c8-ebd4d18ace94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:38.885133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ccb7f-1036-4f72-81e0-9096c0566f88 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:38.902844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:38.915122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec031765-46b3-4842-94c8-ebd4d18ace94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:38.917200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec031765-46b3-4842-94c8-ebd4d18ace94 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:38.921941 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31b0bf-2d92-4562-954c-1f9d0f64b110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:38.922611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31b0bf-2d92-4562-954c-1f9d0f64b110 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:38.922893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31b0bf-2d92-4562-954c-1f9d0f64b110 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:38.923164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31b0bf-2d92-4562-954c-1f9d0f64b110 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:38.935706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d31b0bf-2d92-4562-954c-1f9d0f64b110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-2d31b0bf-2d92-4562-954c-1f9d0f64b110 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:38.940908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:38.941837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:38.942743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:38.942743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:38.942743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:38.942743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:38.942743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:38.942743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:38.942743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:38.942743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:38.942743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:38.942743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:38.942743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:38.942743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:38.943447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:38.950563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 271/555] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:38.950563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31b0bf-2d92-4562-954c-1f9d0f64b110 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:39.041695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 285/556] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 12:37:39.042617 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec031765-46b3-4842-94c8-ebd4d18ace94 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:39.079815 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31b0bf-2d92-4562-954c-1f9d0f64b110 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:39.124509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef70c42-5e81-4b4f-91f2-a552fedcd1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:39.125505 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70c42-5e81-4b4f-91f2-a552fedcd1ad None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.125865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70c42-5e81-4b4f-91f2-a552fedcd1ad None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.126427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70c42-5e81-4b4f-91f2-a552fedcd1ad None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.127144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef70c42-5e81-4b4f-91f2-a552fedcd1ad None admin] RBAC: Adding query filter params (name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:39.127433 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef70c42-5e81-4b4f-91f2-a552fedcd1ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:39.129495 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef70c42-5e81-4b4f-91f2-a552fedcd1ad None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:39.141639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 272/557] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:39.142513 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef70c42-5e81-4b4f-91f2-a552fedcd1ad None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:39.150640 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec031765-46b3-4842-94c8-ebd4d18ace94 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:39.173340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:39.175390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:39.189763 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc3b6d-64e7-4836-80e3-8f97f4b79778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:39.190367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc3b6d-64e7-4836-80e3-8f97f4b79778 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.198984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc3b6d-64e7-4836-80e3-8f97f4b79778 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.198984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc3b6d-64e7-4836-80e3-8f97f4b79778 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.198984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc3b6d-64e7-4836-80e3-8f97f4b79778 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:39.198984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc3b6d-64e7-4836-80e3-8f97f4b79778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:39.198984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc3b6d-64e7-4836-80e3-8f97f4b79778 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:39.201722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 286/558] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:39.203002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e71e95-f730-467f-8ed1-1f4866f27da8 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.203288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e71e95-f730-467f-8ed1-1f4866f27da8 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.203555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e71e95-f730-467f-8ed1-1f4866f27da8 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.206842 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 287/559] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:39.207759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e71e95-f730-467f-8ed1-1f4866f27da8 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:39.253029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef70c42-5e81-4b4f-91f2-a552fedcd1ad None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:39.289515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe1c58-7af3-44bd-aac2-ec9600f7cac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:39.290147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe1c58-7af3-44bd-aac2-ec9600f7cac5 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.290423 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe1c58-7af3-44bd-aac2-ec9600f7cac5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.290764 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe1c58-7af3-44bd-aac2-ec9600f7cac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.301426 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe1c58-7af3-44bd-aac2-ec9600f7cac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:39.302826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe1c58-7af3-44bd-aac2-ec9600f7cac5 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-55fe1c58-7af3-44bd-aac2-ec9600f7cac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:39.317567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:39.318567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:39.319483 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:39.319483 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:39.319483 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:39.319877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e71e95-f730-467f-8ed1-1f4866f27da8 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:39.323384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 273/560] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:39.324260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe1c58-7af3-44bd-aac2-ec9600f7cac5 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:39.355618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa039941-5819-40f1-9d84-3b680ec590a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:39.356584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa039941-5819-40f1-9d84-3b680ec590a6 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.357084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa039941-5819-40f1-9d84-3b680ec590a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.357444 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa039941-5819-40f1-9d84-3b680ec590a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.369900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa039941-5819-40f1-9d84-3b680ec590a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:39.371409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa039941-5819-40f1-9d84-3b680ec590a6 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-fa039941-5819-40f1-9d84-3b680ec590a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:39.386032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:39.387070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:39.387962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:39.387962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:39.387962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:39.387962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 288/561] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:39.388366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc1acd-3c8c-4af3-a447-24bf6662d677 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.388562 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc1acd-3c8c-4af3-a447-24bf6662d677 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.388792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc1acd-3c8c-4af3-a447-24bf6662d677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.405953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-82dc1acd-3c8c-4af3-a447-24bf6662d677 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:39.407163 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-82dc1acd-3c8c-4af3-a447-24bf6662d677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:39.411492 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:39.426633 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe1c58-7af3-44bd-aac2-ec9600f7cac5 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:39.432844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:39.460748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69204039-7ee7-4b28-9844-8603aa71ddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:39.461520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69204039-7ee7-4b28-9844-8603aa71ddd1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.461863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69204039-7ee7-4b28-9844-8603aa71ddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.462132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69204039-7ee7-4b28-9844-8603aa71ddd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.471644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 289/562] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:39.472283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69204039-7ee7-4b28-9844-8603aa71ddd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:39.472456 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc1acd-3c8c-4af3-a447-24bf6662d677 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:39.473900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69204039-7ee7-4b28-9844-8603aa71ddd1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-69204039-7ee7-4b28-9844-8603aa71ddd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:39.490546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:39.492076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:39.492967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:39.492967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:39.492967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:39.492967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 274/563] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:39.493621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314113f4-b3d6-4df1-bfd4-2b1a407536b9 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.493782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314113f4-b3d6-4df1-bfd4-2b1a407536b9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.494044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314113f4-b3d6-4df1-bfd4-2b1a407536b9 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.498549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 275/564] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:39.498549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314113f4-b3d6-4df1-bfd4-2b1a407536b9 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:39.581200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc1acd-3c8c-4af3-a447-24bf6662d677 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:39.604954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314113f4-b3d6-4df1-bfd4-2b1a407536b9 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:39.616324 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac9f61-4f8a-4bfc-afa5-c671181d3cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:39.616977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac9f61-4f8a-4bfc-afa5-c671181d3cfa None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.617252 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac9f61-4f8a-4bfc-afa5-c671181d3cfa None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.617521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac9f61-4f8a-4bfc-afa5-c671181d3cfa None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.618163 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac9f61-4f8a-4bfc-afa5-c671181d3cfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:39.618461 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac9f61-4f8a-4bfc-afa5-c671181d3cfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:39.619835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac9f61-4f8a-4bfc-afa5-c671181d3cfa None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:39.630903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 290/565] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:39.631524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac9f61-4f8a-4bfc-afa5-c671181d3cfa None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:39.644034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac09e4-6a19-417f-8dfd-f6a26cdad2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:39.644638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac09e4-6a19-417f-8dfd-f6a26cdad2e2 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.644915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac09e4-6a19-417f-8dfd-f6a26cdad2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.645181 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac09e4-6a19-417f-8dfd-f6a26cdad2e2 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.645809 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac09e4-6a19-417f-8dfd-f6a26cdad2e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:39.646106 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac09e4-6a19-417f-8dfd-f6a26cdad2e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:39.647524 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac09e4-6a19-417f-8dfd-f6a26cdad2e2 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:39.649811 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:39.659045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:39.665611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 276/566] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:39.665611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8d2e5-de10-44ce-beac-fb88082bf3f5 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.665611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8d2e5-de10-44ce-beac-fb88082bf3f5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.665611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8d2e5-de10-44ce-beac-fb88082bf3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.682186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-d6a8d2e5-de10-44ce-beac-fb88082bf3f5 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:39.685533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d6a8d2e5-de10-44ce-beac-fb88082bf3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:39.743326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac9f61-4f8a-4bfc-afa5-c671181d3cfa None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:39.759221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 277/567] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:39.759901 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8d2e5-de10-44ce-beac-fb88082bf3f5 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:39.778507 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb6324-9ee0-428e-a234-25de6b457942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:39.779217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb6324-9ee0-428e-a234-25de6b457942 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.779564 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb6324-9ee0-428e-a234-25de6b457942 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.779902 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb6324-9ee0-428e-a234-25de6b457942 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.780610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb6324-9ee0-428e-a234-25de6b457942 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:39.780980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb6324-9ee0-428e-a234-25de6b457942 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:39.782389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb6324-9ee0-428e-a234-25de6b457942 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:39.795418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 291/568] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:39.796131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb6324-9ee0-428e-a234-25de6b457942 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:39.868584 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8d2e5-de10-44ce-beac-fb88082bf3f5 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:39.882026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:39.895174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:39.897346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb6324-9ee0-428e-a234-25de6b457942 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:39.904203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f422e18-cd2e-463a-aa64-082f804afa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:39.904836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f422e18-cd2e-463a-aa64-082f804afa40 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.905116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f422e18-cd2e-463a-aa64-082f804afa40 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.905388 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f422e18-cd2e-463a-aa64-082f804afa40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.912301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f422e18-cd2e-463a-aa64-082f804afa40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:39.914348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f422e18-cd2e-463a-aa64-082f804afa40 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-4f422e18-cd2e-463a-aa64-082f804afa40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:39.925853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:39.927075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:39.928107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:39.928107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:39.928107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:39.928107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:39.928107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:39.928107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:39.928107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:39.928107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:39.928107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 278/569] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:39.931816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f422e18-cd2e-463a-aa64-082f804afa40 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:39.936376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c4fbd-ce70-4075-8aab-be2299282d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:39.937007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c4fbd-ce70-4075-8aab-be2299282d79 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:39.937293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c4fbd-ce70-4075-8aab-be2299282d79 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:39.937567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c4fbd-ce70-4075-8aab-be2299282d79 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:39.944892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c4fbd-ce70-4075-8aab-be2299282d79 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:39.946605 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c4fbd-ce70-4075-8aab-be2299282d79 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-bb6c4fbd-ce70-4075-8aab-be2299282d79 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:39.958334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:39.964233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:39.965405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:39.965405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:39.965405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:39.965405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:39.965405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:39.965405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:39.965405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:39.965405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:39.965405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:37:39.965405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:39.965405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 292/570] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:39.965405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c4fbd-ce70-4075-8aab-be2299282d79 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:40.044907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f422e18-cd2e-463a-aa64-082f804afa40 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:40.072144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c4fbd-ce70-4075-8aab-be2299282d79 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:40.078177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333bd9a-de14-4a85-96fd-c8cf08ae4f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:40.078836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333bd9a-de14-4a85-96fd-c8cf08ae4f10 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:40.079117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333bd9a-de14-4a85-96fd-c8cf08ae4f10 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:40.079387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333bd9a-de14-4a85-96fd-c8cf08ae4f10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:40.089859 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333bd9a-de14-4a85-96fd-c8cf08ae4f10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-0333bd9a-de14-4a85-96fd-c8cf08ae4f10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:40.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:40.096115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 279/571] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:40.099451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333bd9a-de14-4a85-96fd-c8cf08ae4f10 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:40.108168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3cb04-4c83-460a-863a-07b87d5d37d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:40.108801 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3cb04-4c83-460a-863a-07b87d5d37d6 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:40.109076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3cb04-4c83-460a-863a-07b87d5d37d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:40.109342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3cb04-4c83-460a-863a-07b87d5d37d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:40.117257 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3cb04-4c83-460a-863a-07b87d5d37d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:40.119238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3cb04-4c83-460a-863a-07b87d5d37d6 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-2dc3cb04-4c83-460a-863a-07b87d5d37d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:40.130212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:40.131377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:40.132784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:40.132784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:40.132784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:40.132784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:40.132784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:40.132784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:40.132784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:40.132784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:40.132784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:37:40.132784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:40.132784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 293/572] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 12:37:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:40.132784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3cb04-4c83-460a-863a-07b87d5d37d6 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:40.136577 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:40.151057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:40.202410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333bd9a-de14-4a85-96fd-c8cf08ae4f10 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:40.233785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3cb04-4c83-460a-863a-07b87d5d37d6 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:40.235409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bc45b-3690-4430-a198-a4a68e03bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:40.236006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bc45b-3690-4430-a198-a4a68e03bc9c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:40.236280 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bc45b-3690-4430-a198-a4a68e03bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:40.236548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bc45b-3690-4430-a198-a4a68e03bc9c None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:40.237218 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020bc45b-3690-4430-a198-a4a68e03bc9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:40.237516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020bc45b-3690-4430-a198-a4a68e03bc9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:40.238968 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020bc45b-3690-4430-a198-a4a68e03bc9c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:40.250351 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 280/573] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:40.251150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bc45b-3690-4430-a198-a4a68e03bc9c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:40.266447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77d72c-999e-462e-85e4-836fa8c9e082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:40.267156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77d72c-999e-462e-85e4-836fa8c9e082 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:40.267432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77d72c-999e-462e-85e4-836fa8c9e082 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:40.267700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77d72c-999e-462e-85e4-836fa8c9e082 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:40.268439 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77d72c-999e-462e-85e4-836fa8c9e082 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:40.268743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77d72c-999e-462e-85e4-836fa8c9e082 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:40.270143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77d72c-999e-462e-85e4-836fa8c9e082 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:40.287656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 294/574] 10.222.0.46 () {56 vars in 1137 bytes} [Fri Sep 9 12:37:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:40.288545 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77d72c-999e-462e-85e4-836fa8c9e082 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:40.349514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bc45b-3690-4430-a198-a4a68e03bc9c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:40.350027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:40.356963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:40.390249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955bca0-2f85-4761-b53e-f6d5f27c5979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:40.390824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955bca0-2f85-4761-b53e-f6d5f27c5979 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:40.391101 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955bca0-2f85-4761-b53e-f6d5f27c5979 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:40.391370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955bca0-2f85-4761-b53e-f6d5f27c5979 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:40.391764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77d72c-999e-462e-85e4-836fa8c9e082 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:40.401520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955bca0-2f85-4761-b53e-f6d5f27c5979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-5955bca0-2f85-4761-b53e-f6d5f27c5979 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:40.406349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:40.407458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:40.408312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:40.408312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:40.408312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:40.408312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:40.408312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:40.408312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:40.408312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:40.408312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:40.408312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:40.408312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:40.408312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:40.408312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:40.408312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 281/575] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:40.410119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955bca0-2f85-4761-b53e-f6d5f27c5979 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:40.428294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf60f6c-4242-4024-8f16-3fb7901bdc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:40.428892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf60f6c-4242-4024-8f16-3fb7901bdc84 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:40.429700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf60f6c-4242-4024-8f16-3fb7901bdc84 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:40.429972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf60f6c-4242-4024-8f16-3fb7901bdc84 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:40.430656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf60f6c-4242-4024-8f16-3fb7901bdc84 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:40.430955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf60f6c-4242-4024-8f16-3fb7901bdc84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:40.432283 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf60f6c-4242-4024-8f16-3fb7901bdc84 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:40.440150 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 295/576] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:40.440935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf60f6c-4242-4024-8f16-3fb7901bdc84 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:40.512686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955bca0-2f85-4761-b53e-f6d5f27c5979 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:40.536761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf60f6c-4242-4024-8f16-3fb7901bdc84 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:40.545628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca25180-5d33-47c5-b6c8-264abdea870a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:40.546626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca25180-5d33-47c5-b6c8-264abdea870a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:40.546909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca25180-5d33-47c5-b6c8-264abdea870a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:40.547191 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca25180-5d33-47c5-b6c8-264abdea870a None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:40.547898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca25180-5d33-47c5-b6c8-264abdea870a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:40.548202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca25180-5d33-47c5-b6c8-264abdea870a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:40.549629 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca25180-5d33-47c5-b6c8-264abdea870a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:40.560177 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:40.566861 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 282/577] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 12:37:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:40.567686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca25180-5d33-47c5-b6c8-264abdea870a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:40.570424 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55720f-f768-4542-b864-ea6e7a022648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:40.571043 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55720f-f768-4542-b864-ea6e7a022648 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:40.571417 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55720f-f768-4542-b864-ea6e7a022648 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:40.577372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55720f-f768-4542-b864-ea6e7a022648 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:40.577372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55720f-f768-4542-b864-ea6e7a022648 None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=e8e4805f0b1e484c98fb399313f88ef2) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:40.577372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55720f-f768-4542-b864-ea6e7a022648 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:40.577372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55720f-f768-4542-b864-ea6e7a022648 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:40.579571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 296/578] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:40 2022] GET /identity/v3/role_assignments?user.id=e8e4805f0b1e484c98fb399313f88ef2&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:40.580329 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55720f-f768-4542-b864-ea6e7a022648 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:40.594061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:40.687657 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca25180-5d33-47c5-b6c8-264abdea870a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:40.691214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55720f-f768-4542-b864-ea6e7a022648 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:40.725457 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06b01f-4f33-4449-a008-1fa6cf608809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:40.726867 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06b01f-4f33-4449-a008-1fa6cf608809 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:40.727228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06b01f-4f33-4449-a008-1fa6cf608809 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:40.727565 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06b01f-4f33-4449-a008-1fa6cf608809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:40.729901 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c07f6-c1e6-4bea-a370-71b824b100c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:40.730627 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c07f6-c1e6-4bea-a370-71b824b100c9 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:40.730980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c07f6-c1e6-4bea-a370-71b824b100c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:40.731303 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c07f6-c1e6-4bea-a370-71b824b100c9 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:40.731975 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c07f6-c1e6-4bea-a370-71b824b100c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:40.732277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c07f6-c1e6-4bea-a370-71b824b100c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:40.733641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c07f6-c1e6-4bea-a370-71b824b100c9 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:40.741043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06b01f-4f33-4449-a008-1fa6cf608809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:40.746123 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06b01f-4f33-4449-a008-1fa6cf608809 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:40.746305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 297/579] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:40.746305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c07f6-c1e6-4bea-a370-71b824b100c9 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-8b06b01f-4f33-4449-a008-1fa6cf608809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:40.765875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:40.767342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:40.769036 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:40.769036 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:40.769036 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:40.770622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 283/580] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:40.771707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06b01f-4f33-4449-a008-1fa6cf608809 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:40.811422 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:40.861777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:40.866465 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c07f6-c1e6-4bea-a370-71b824b100c9 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:40.886283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06b01f-4f33-4449-a008-1fa6cf608809 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:40.902243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8388d14-a8e4-4cfd-9ad3-77b9ead42798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:40.902900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8388d14-a8e4-4cfd-9ad3-77b9ead42798 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:40.903172 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8388d14-a8e4-4cfd-9ad3-77b9ead42798 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:40.903501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8388d14-a8e4-4cfd-9ad3-77b9ead42798 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:40.914634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8388d14-a8e4-4cfd-9ad3-77b9ead42798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:40.916073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8388d14-a8e4-4cfd-9ad3-77b9ead42798 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:40.921765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f59e8-23be-42fa-ba17-3dcab66bc4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:40.922403 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f59e8-23be-42fa-ba17-3dcab66bc4d6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:40.922708 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f59e8-23be-42fa-ba17-3dcab66bc4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:40.922983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f59e8-23be-42fa-ba17-3dcab66bc4d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-f8388d14-a8e4-4cfd-9ad3-77b9ead42798 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:40.931077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:40.932034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:40.932938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:40.932938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:40.932938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:40.932938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 298/581] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:40.933666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8388d14-a8e4-4cfd-9ad3-77b9ead42798 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:40.938859 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f59e8-23be-42fa-ba17-3dcab66bc4d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:40.940273 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f59e8-23be-42fa-ba17-3dcab66bc4d6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-157f59e8-23be-42fa-ba17-3dcab66bc4d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:40.957320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:40.958538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:40.959669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:40.959669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:40.959669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:40.963519 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 284/582] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:40.964339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f59e8-23be-42fa-ba17-3dcab66bc4d6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:41.041626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8388d14-a8e4-4cfd-9ad3-77b9ead42798 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:41.043142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:41.074938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f59e8-23be-42fa-ba17-3dcab66bc4d6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:41.076808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d39906-3321-4a48-ae41-1f2a8388053b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:41.078270 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d39906-3321-4a48-ae41-1f2a8388053b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:41.078601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d39906-3321-4a48-ae41-1f2a8388053b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:41.078946 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d39906-3321-4a48-ae41-1f2a8388053b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:41.093413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d39906-3321-4a48-ae41-1f2a8388053b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:41.095747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d39906-3321-4a48-ae41-1f2a8388053b None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:41.107004 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-46d39906-3321-4a48-ae41-1f2a8388053b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:41.114767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:41.115807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:41.116971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:41.116971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:41.116971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:41.116971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 299/583] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:41.116971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d39906-3321-4a48-ae41-1f2a8388053b None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:41.125855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f15f63-6f18-45c0-995e-4444d326e4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:41.126564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f15f63-6f18-45c0-995e-4444d326e4c6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:41.126866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f15f63-6f18-45c0-995e-4444d326e4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:41.127206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f15f63-6f18-45c0-995e-4444d326e4c6 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:41.127873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f15f63-6f18-45c0-995e-4444d326e4c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:41.128180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f15f63-6f18-45c0-995e-4444d326e4c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:41.129596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f15f63-6f18-45c0-995e-4444d326e4c6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:41.154990 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 285/584] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:41.156146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f15f63-6f18-45c0-995e-4444d326e4c6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:41.228502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d39906-3321-4a48-ae41-1f2a8388053b None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:41.260428 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f15f63-6f18-45c0-995e-4444d326e4c6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:41.263339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45743099-ce75-4dae-9965-4adcb29014b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:41.263947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45743099-ce75-4dae-9965-4adcb29014b6 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:41.264227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45743099-ce75-4dae-9965-4adcb29014b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:41.264499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45743099-ce75-4dae-9965-4adcb29014b6 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:41.265142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45743099-ce75-4dae-9965-4adcb29014b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:41.265450 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45743099-ce75-4dae-9965-4adcb29014b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:41.266837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45743099-ce75-4dae-9965-4adcb29014b6 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:41.279180 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 300/585] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:41.280012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45743099-ce75-4dae-9965-4adcb29014b6 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:41.297036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:41.303817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04456d83-9304-459b-94df-47cbc73f0882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:41.304815 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04456d83-9304-459b-94df-47cbc73f0882 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:41.309507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04456d83-9304-459b-94df-47cbc73f0882 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:41.309507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04456d83-9304-459b-94df-47cbc73f0882 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:41.309507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04456d83-9304-459b-94df-47cbc73f0882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:41.309507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04456d83-9304-459b-94df-47cbc73f0882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:41.309507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04456d83-9304-459b-94df-47cbc73f0882 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:41.320310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 286/586] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:41.325367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04456d83-9304-459b-94df-47cbc73f0882 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:41.378401 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:41.386252 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45743099-ce75-4dae-9965-4adcb29014b6 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:41.420316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afcae1f-6658-4820-9d59-a7fa85ad162d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:41.420919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcae1f-6658-4820-9d59-a7fa85ad162d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:41.421192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcae1f-6658-4820-9d59-a7fa85ad162d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:41.421458 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcae1f-6658-4820-9d59-a7fa85ad162d None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:41.422090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afcae1f-6658-4820-9d59-a7fa85ad162d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:41.422385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afcae1f-6658-4820-9d59-a7fa85ad162d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:41.423749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afcae1f-6658-4820-9d59-a7fa85ad162d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:41.433456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04456d83-9304-459b-94df-47cbc73f0882 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:41.434802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 301/587] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:41.435887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afcae1f-6658-4820-9d59-a7fa85ad162d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:41.481280 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31081ce-2c3a-441c-89a9-6fe798e2d764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:41.482142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31081ce-2c3a-441c-89a9-6fe798e2d764 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:41.482714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31081ce-2c3a-441c-89a9-6fe798e2d764 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:41.482992 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31081ce-2c3a-441c-89a9-6fe798e2d764 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:41.489980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31081ce-2c3a-441c-89a9-6fe798e2d764 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:41.492106 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31081ce-2c3a-441c-89a9-6fe798e2d764 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-f31081ce-2c3a-441c-89a9-6fe798e2d764 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:41.504251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:41.505126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:41.506034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:41.506034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:41.506034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:41.506034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:41.506034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:41.506034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:41.506034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:41.506034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:41.506034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 287/588] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:41.506559 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31081ce-2c3a-441c-89a9-6fe798e2d764 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:41.520277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:41.546931 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afcae1f-6658-4820-9d59-a7fa85ad162d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:41.581690 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908e2c6-84ce-481d-a6a3-c46a5b4125c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:41.582317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908e2c6-84ce-481d-a6a3-c46a5b4125c2 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:41.582626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908e2c6-84ce-481d-a6a3-c46a5b4125c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:41.582901 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908e2c6-84ce-481d-a6a3-c46a5b4125c2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:41.591667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908e2c6-84ce-481d-a6a3-c46a5b4125c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:41.593098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908e2c6-84ce-481d-a6a3-c46a5b4125c2 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-f908e2c6-84ce-481d-a6a3-c46a5b4125c2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:41.605210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:41.606226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:41.607517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:41.607517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:41.607517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:41.607517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:41.607517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:41.607517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:41.607517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:41.607517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:41.607517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 12:37:41.607517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:41.607517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 302/589] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:41.607517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908e2c6-84ce-481d-a6a3-c46a5b4125c2 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:41.619027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31081ce-2c3a-441c-89a9-6fe798e2d764 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:41.627751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:41.652253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30348b3e-39c4-4d0c-bf2a-85a66d8082d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:41.652872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30348b3e-39c4-4d0c-bf2a-85a66d8082d9 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:41.653151 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30348b3e-39c4-4d0c-bf2a-85a66d8082d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:41.653417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30348b3e-39c4-4d0c-bf2a-85a66d8082d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:41.659922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30348b3e-39c4-4d0c-bf2a-85a66d8082d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:41.661345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30348b3e-39c4-4d0c-bf2a-85a66d8082d9 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-30348b3e-39c4-4d0c-bf2a-85a66d8082d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:41.673949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:41.674960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:41.681031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:41.681031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:41.681031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:41.681031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:41.681031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:41.681031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:41.681031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:41.681031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:41.681031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 288/590] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:41.681031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30348b3e-39c4-4d0c-bf2a-85a66d8082d9 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:41.728874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908e2c6-84ce-481d-a6a3-c46a5b4125c2 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:41.763391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762e817-f0e4-4a42-887e-305205a18181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:41.764000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762e817-f0e4-4a42-887e-305205a18181 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:41.764190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762e817-f0e4-4a42-887e-305205a18181 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:41.764454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762e817-f0e4-4a42-887e-305205a18181 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:41.766426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:41.772382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762e817-f0e4-4a42-887e-305205a18181 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:41.774345 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762e817-f0e4-4a42-887e-305205a18181 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-2762e817-f0e4-4a42-887e-305205a18181 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:41.785102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:41.786038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:41.787014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:41.787014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:41.787014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:41.787014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:41.787014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:41.787014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:41.787014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:41.787014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:41.787014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 12:37:41.787014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:41.787014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 303/591] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:41.787014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762e817-f0e4-4a42-887e-305205a18181 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:41.877779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30348b3e-39c4-4d0c-bf2a-85a66d8082d9 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:41.911585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff30e48-594c-493a-977a-df892ea0d0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:41.912190 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff30e48-594c-493a-977a-df892ea0d0e8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:41.912468 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff30e48-594c-493a-977a-df892ea0d0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:41.912741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff30e48-594c-493a-977a-df892ea0d0e8 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:41.913135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762e817-f0e4-4a42-887e-305205a18181 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:41.913443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff30e48-594c-493a-977a-df892ea0d0e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:41.913765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff30e48-594c-493a-977a-df892ea0d0e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:41.915223 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff30e48-594c-493a-977a-df892ea0d0e8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:41.931749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 289/592] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:41.931749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff30e48-594c-493a-977a-df892ea0d0e8 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:41.954327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eb31e-080c-4298-a85c-66ce97b3f607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:41.955036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb31e-080c-4298-a85c-66ce97b3f607 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:41.955373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb31e-080c-4298-a85c-66ce97b3f607 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:41.955714 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb31e-080c-4298-a85c-66ce97b3f607 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:41.956499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eb31e-080c-4298-a85c-66ce97b3f607 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:41.956870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eb31e-080c-4298-a85c-66ce97b3f607 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:41.958279 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eb31e-080c-4298-a85c-66ce97b3f607 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:41.960104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:41.980464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 304/593] 10.222.0.46 () {56 vars in 1137 bytes} [Fri Sep 9 12:37:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:41.980464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eb31e-080c-4298-a85c-66ce97b3f607 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:42.034838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:42.039104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff30e48-594c-493a-977a-df892ea0d0e8 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:42.078064 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead96856-2864-485a-b6af-b8d3e698bb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:42.078703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead96856-2864-485a-b6af-b8d3e698bb14 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:42.078984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead96856-2864-485a-b6af-b8d3e698bb14 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:42.079255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead96856-2864-485a-b6af-b8d3e698bb14 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:42.079943 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead96856-2864-485a-b6af-b8d3e698bb14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:42.080246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead96856-2864-485a-b6af-b8d3e698bb14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:42.081688 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead96856-2864-485a-b6af-b8d3e698bb14 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:42.092871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eb31e-080c-4298-a85c-66ce97b3f607 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:42.095430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 290/594] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:42.096241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead96856-2864-485a-b6af-b8d3e698bb14 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:42.140317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec5895-ed09-45e0-9832-a06f2caf9fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:42.140928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec5895-ed09-45e0-9832-a06f2caf9fdd None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:42.141205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec5895-ed09-45e0-9832-a06f2caf9fdd None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:42.141477 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec5895-ed09-45e0-9832-a06f2caf9fdd None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:42.142203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec5895-ed09-45e0-9832-a06f2caf9fdd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:42.142525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec5895-ed09-45e0-9832-a06f2caf9fdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:42.143933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec5895-ed09-45e0-9832-a06f2caf9fdd None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:42.163947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 305/595] 10.222.0.46 () {56 vars in 1139 bytes} [Fri Sep 9 12:37:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:42.164600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec5895-ed09-45e0-9832-a06f2caf9fdd None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:42.206939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:42.220065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead96856-2864-485a-b6af-b8d3e698bb14 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:42.257028 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b40bab-953f-4da1-b247-fb4ae4aa9e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:42.258860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b40bab-953f-4da1-b247-fb4ae4aa9e8a None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:42.259143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b40bab-953f-4da1-b247-fb4ae4aa9e8a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:42.259420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b40bab-953f-4da1-b247-fb4ae4aa9e8a None admin] PATH_INFO: `/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/002c117a85c1452e9fae80c755c6622f/roles/4a5e4b92c86041c6aa22c4f4735b4a05` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:42.270155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec5895-ed09-45e0-9832-a06f2caf9fdd None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:42.275826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b40bab-953f-4da1-b247-fb4ae4aa9e8a None admin] RBAC: Authorizing `identity:create_grant(project_id=6e2dc716277d42ae8316ebb2d36293a8, user_id=002c117a85c1452e9fae80c755c6622f, role_id=4a5e4b92c86041c6aa22c4f4735b4a05)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:42.277261 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b40bab-953f-4da1-b247-fb4ae4aa9e8a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:42.282534 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:42.305525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f399e0a-e506-4ef3-b116-c1afccce5b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:42.306080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f399e0a-e506-4ef3-b116-c1afccce5b68 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:42.306356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f399e0a-e506-4ef3-b116-c1afccce5b68 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:42.306677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f399e0a-e506-4ef3-b116-c1afccce5b68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:42.316859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f399e0a-e506-4ef3-b116-c1afccce5b68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:42.318295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f399e0a-e506-4ef3-b116-c1afccce5b68 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-9f399e0a-e506-4ef3-b116-c1afccce5b68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:42.333803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:42.340323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:42.341418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:42.341418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:42.341418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:42.341418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 306/596] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:42.341418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f399e0a-e506-4ef3-b116-c1afccce5b68 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:42.347625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 291/597] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:37:42 2022] PUT /identity/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/002c117a85c1452e9fae80c755c6622f/roles/4a5e4b92c86041c6aa22c4f4735b4a05 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:37:42.349659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b40bab-953f-4da1-b247-fb4ae4aa9e8a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:42.476188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b40bab-953f-4da1-b247-fb4ae4aa9e8a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:42.490324 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f399e0a-e506-4ef3-b116-c1afccce5b68 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:42.521057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3e244-baaf-4289-ae00-148e8cac052d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:42.522197 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3e244-baaf-4289-ae00-148e8cac052d None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:42.522696 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3e244-baaf-4289-ae00-148e8cac052d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:42.523779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3e244-baaf-4289-ae00-148e8cac052d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:42.532099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:42.537768 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3e244-baaf-4289-ae00-148e8cac052d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:42.538128 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04029d7a-463f-492b-8a3c-f8c000a31e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:42.539432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04029d7a-463f-492b-8a3c-f8c000a31e40 None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:42.539709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04029d7a-463f-492b-8a3c-f8c000a31e40 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:42.540072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04029d7a-463f-492b-8a3c-f8c000a31e40 None admin] PATH_INFO: `/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/f29af98c5fba4569b16edf240b13c5c5/roles/4a5e4b92c86041c6aa22c4f4735b4a05` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:42.541530 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3e244-baaf-4289-ae00-148e8cac052d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:42.557137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04029d7a-463f-492b-8a3c-f8c000a31e40 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e2dc716277d42ae8316ebb2d36293a8, user_id=f29af98c5fba4569b16edf240b13c5c5, role_id=4a5e4b92c86041c6aa22c4f4735b4a05)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-c2b3e244-baaf-4289-ae00-148e8cac052d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:42.558240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:42.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:42.560730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:42.560730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:42.560730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:42.560912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04029d7a-463f-492b-8a3c-f8c000a31e40 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:42.560997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 292/598] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:42.562515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3e244-baaf-4289-ae00-148e8cac052d None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:42.646420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 307/599] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:37:42 2022] PUT /identity/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/f29af98c5fba4569b16edf240b13c5c5/roles/4a5e4b92c86041c6aa22c4f4735b4a05 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:37:42.647325 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04029d7a-463f-492b-8a3c-f8c000a31e40 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:42.676821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:42.712202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3e244-baaf-4289-ae00-148e8cac052d None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:42.780643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028dca6-4c9b-4ca5-925e-3c58d80f3257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:42.781837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028dca6-4c9b-4ca5-925e-3c58d80f3257 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:42.783083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028dca6-4c9b-4ca5-925e-3c58d80f3257 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:42.783201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028dca6-4c9b-4ca5-925e-3c58d80f3257 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:42.783815 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028dca6-4c9b-4ca5-925e-3c58d80f3257 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:42.784122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028dca6-4c9b-4ca5-925e-3c58d80f3257 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:42.785596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028dca6-4c9b-4ca5-925e-3c58d80f3257 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:42.800468 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 293/600] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:42.801769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5f50d-2b00-488c-99c6-fd39423ecd86 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:42.802058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5f50d-2b00-488c-99c6-fd39423ecd86 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:42.802335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5f50d-2b00-488c-99c6-fd39423ecd86 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:42.808653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 294/601] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:42.808653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5f50d-2b00-488c-99c6-fd39423ecd86 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:42.835213 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04029d7a-463f-492b-8a3c-f8c000a31e40 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:42.878889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f53744c-f34b-4511-bc82-5a703e32b20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:42.879768 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53744c-f34b-4511-bc82-5a703e32b20c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:42.880050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53744c-f34b-4511-bc82-5a703e32b20c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:42.880688 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:42.881340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53744c-f34b-4511-bc82-5a703e32b20c None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:42.882008 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f53744c-f34b-4511-bc82-5a703e32b20c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:42.882365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f53744c-f34b-4511-bc82-5a703e32b20c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:42.884191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f53744c-f34b-4511-bc82-5a703e32b20c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:42.899525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 308/602] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:42.901671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96b0f4-bcc4-4958-bdab-6772d266b0a7 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:42.902154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96b0f4-bcc4-4958-bdab-6772d266b0a7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:42.902671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96b0f4-bcc4-4958-bdab-6772d266b0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:42.926572 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0d96b0f4-bcc4-4958-bdab-6772d266b0a7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:42.928117 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0d96b0f4-bcc4-4958-bdab-6772d266b0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:42.943281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5f50d-2b00-488c-99c6-fd39423ecd86 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:42.989238 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e48d20-9602-442c-9f9a-ddff7fd5624e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:42.989960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e48d20-9602-442c-9f9a-ddff7fd5624e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:42.990237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e48d20-9602-442c-9f9a-ddff7fd5624e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:42.990568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e48d20-9602-442c-9f9a-ddff7fd5624e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:42.998920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e48d20-9602-442c-9f9a-ddff7fd5624e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:43.000511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e48d20-9602-442c-9f9a-ddff7fd5624e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:43.012525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 309/603] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-c3e48d20-9602-442c-9f9a-ddff7fd5624e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:43.012910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:43.013955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:43.015184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:43.015184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:43.015184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:43.015184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:43.015184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:43.015184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:43.015184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:43.015184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:43.015184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 295/604] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:43.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96b0f4-bcc4-4958-bdab-6772d266b0a7 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:43.019027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:43.020773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e48d20-9602-442c-9f9a-ddff7fd5624e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:43.130900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96b0f4-bcc4-4958-bdab-6772d266b0a7 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:43.191550 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd558791-ba6b-4d3b-b530-d47aeeaf4dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:43.193129 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd558791-ba6b-4d3b-b530-d47aeeaf4dcd None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:43.193406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd558791-ba6b-4d3b-b530-d47aeeaf4dcd None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:43.193675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd558791-ba6b-4d3b-b530-d47aeeaf4dcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:43.207547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e48d20-9602-442c-9f9a-ddff7fd5624e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:43.209950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd558791-ba6b-4d3b-b530-d47aeeaf4dcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:43.212089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd558791-ba6b-4d3b-b530-d47aeeaf4dcd None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:43.230844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-cd558791-ba6b-4d3b-b530-d47aeeaf4dcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:43.234625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:43.235496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:43.236378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:43.236378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:43.236378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:43.236378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:43.236378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:43.236378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:43.236378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:43.236378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:43.242524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 310/605] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:43.248248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd558791-ba6b-4d3b-b530-d47aeeaf4dcd None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:43.276507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fa8c0-cf48-4342-bd44-f4f8cbf48b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:43.277132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fa8c0-cf48-4342-bd44-f4f8cbf48b48 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:43.279615 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fa8c0-cf48-4342-bd44-f4f8cbf48b48 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:43.279896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fa8c0-cf48-4342-bd44-f4f8cbf48b48 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:43.280583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fa8c0-cf48-4342-bd44-f4f8cbf48b48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:43.280885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fa8c0-cf48-4342-bd44-f4f8cbf48b48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:43.282308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fa8c0-cf48-4342-bd44-f4f8cbf48b48 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:43.310659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 296/606] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:43.314181 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fa8c0-cf48-4342-bd44-f4f8cbf48b48 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:43.350544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:43.395493 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd558791-ba6b-4d3b-b530-d47aeeaf4dcd None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:43.433337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ee134-797d-452e-a39f-e18812adbb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:43.434026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ee134-797d-452e-a39f-e18812adbb9b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:43.434399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ee134-797d-452e-a39f-e18812adbb9b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:43.434698 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ee134-797d-452e-a39f-e18812adbb9b None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:43.435654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ee134-797d-452e-a39f-e18812adbb9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:43.435953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ee134-797d-452e-a39f-e18812adbb9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:43.437609 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ee134-797d-452e-a39f-e18812adbb9b None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:43.440942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fa8c0-cf48-4342-bd44-f4f8cbf48b48 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:43.456278 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 311/607] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:43.457276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ee134-797d-452e-a39f-e18812adbb9b None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:43.481133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18d5f9-a6bf-4789-8e9c-5ecc21a1d53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:43.482311 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18d5f9-a6bf-4789-8e9c-5ecc21a1d53d None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:43.482625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18d5f9-a6bf-4789-8e9c-5ecc21a1d53d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:43.482900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18d5f9-a6bf-4789-8e9c-5ecc21a1d53d None admin] PATH_INFO: `/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/eff8fcb7fa2740bbb801badf800d2d6a/roles/4a5e4b92c86041c6aa22c4f4735b4a05` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:43.502401 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18d5f9-a6bf-4789-8e9c-5ecc21a1d53d None admin] RBAC: Authorizing `identity:create_grant(project_id=6e2dc716277d42ae8316ebb2d36293a8, user_id=eff8fcb7fa2740bbb801badf800d2d6a, role_id=4a5e4b92c86041c6aa22c4f4735b4a05)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:43.504319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18d5f9-a6bf-4789-8e9c-5ecc21a1d53d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:43.566120 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ee134-797d-452e-a39f-e18812adbb9b None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:43.589804 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:43.606392 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d5d83-9a04-433b-85a1-7964aee0ed43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:43.606392 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d5d83-9a04-433b-85a1-7964aee0ed43 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:43.606392 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d5d83-9a04-433b-85a1-7964aee0ed43 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:43.606392 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d5d83-9a04-433b-85a1-7964aee0ed43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:43.607213 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 297/608] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:37:43 2022] PUT /identity/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/eff8fcb7fa2740bbb801badf800d2d6a/roles/4a5e4b92c86041c6aa22c4f4735b4a05 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:37:43.608712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18d5f9-a6bf-4789-8e9c-5ecc21a1d53d None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:43.630717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d5d83-9a04-433b-85a1-7964aee0ed43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-6b7d5d83-9a04-433b-85a1-7964aee0ed43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:43.637722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:43.639054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:43.640072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:43.640072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:43.640072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:43.640072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:43.640072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:43.640072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:43.640072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:43.640072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:43.640072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:43.640072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:43.640072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:43.640072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:43.647139 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 312/609] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:43.648500 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:43.654404 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d5d83-9a04-433b-85a1-7964aee0ed43 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:43.746883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18d5f9-a6bf-4789-8e9c-5ecc21a1d53d None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:43.785255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d6e43-7a72-4c77-b8d2-d1c560daa366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:43.786371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d6e43-7a72-4c77-b8d2-d1c560daa366 None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:43.786883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d6e43-7a72-4c77-b8d2-d1c560daa366 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:43.787347 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d6e43-7a72-4c77-b8d2-d1c560daa366 None admin] PATH_INFO: `/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/e46bd9d49bf64013ae58770b0284d81e/roles/4a5e4b92c86041c6aa22c4f4735b4a05` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:43.816705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d6e43-7a72-4c77-b8d2-d1c560daa366 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e2dc716277d42ae8316ebb2d36293a8, user_id=e46bd9d49bf64013ae58770b0284d81e, role_id=4a5e4b92c86041c6aa22c4f4735b4a05)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:43.818160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d6e43-7a72-4c77-b8d2-d1c560daa366 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:43.880620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 298/610] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:37:43 2022] PUT /identity/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/e46bd9d49bf64013ae58770b0284d81e/roles/4a5e4b92c86041c6aa22c4f4735b4a05 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:37:43.881364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d5d83-9a04-433b-85a1-7964aee0ed43 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:43.925928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be796a2d-1055-4424-baf8-752b4bad0858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:43.926465 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be796a2d-1055-4424-baf8-752b4bad0858 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:43.926831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be796a2d-1055-4424-baf8-752b4bad0858 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:43.927105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be796a2d-1055-4424-baf8-752b4bad0858 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:43.927838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be796a2d-1055-4424-baf8-752b4bad0858 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:43.928141 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be796a2d-1055-4424-baf8-752b4bad0858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:43.930318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be796a2d-1055-4424-baf8-752b4bad0858 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:43.940494 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 313/611] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:43.946043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d6e43-7a72-4c77-b8d2-d1c560daa366 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:44.024150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:44.065640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d6e43-7a72-4c77-b8d2-d1c560daa366 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:44.105937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cbe2a-08b2-4f5c-b6d5-6ab84539975d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:44.107929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cbe2a-08b2-4f5c-b6d5-6ab84539975d None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:44.107929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cbe2a-08b2-4f5c-b6d5-6ab84539975d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:44.107929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cbe2a-08b2-4f5c-b6d5-6ab84539975d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:44.119312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cbe2a-08b2-4f5c-b6d5-6ab84539975d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:44.120777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cbe2a-08b2-4f5c-b6d5-6ab84539975d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-bc2cbe2a-08b2-4f5c-b6d5-6ab84539975d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:44.138396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:44.139607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:44.140709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:44.140709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:44.140709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:44.140709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 299/612] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:37:44.146558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be796a2d-1055-4424-baf8-752b4bad0858 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:44.193942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:44.250472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be796a2d-1055-4424-baf8-752b4bad0858 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:44.284880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf19809-b8fa-4349-a607-57afe6676ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:44.285576 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf19809-b8fa-4349-a607-57afe6676ee5 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:44.285919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf19809-b8fa-4349-a607-57afe6676ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:44.286253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf19809-b8fa-4349-a607-57afe6676ee5 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:44.286989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf19809-b8fa-4349-a607-57afe6676ee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:44.287351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf19809-b8fa-4349-a607-57afe6676ee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:44.289362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf19809-b8fa-4349-a607-57afe6676ee5 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:44.306179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 314/613] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:44.312272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cbe2a-08b2-4f5c-b6d5-6ab84539975d None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:44.437230 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cbe2a-08b2-4f5c-b6d5-6ab84539975d None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:44.461791 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:44.473402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa234e-4160-48a6-93da-06549de8988f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:44.474116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa234e-4160-48a6-93da-06549de8988f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:44.474465 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa234e-4160-48a6-93da-06549de8988f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:44.474827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa234e-4160-48a6-93da-06549de8988f None admin] PATH_INFO: `/v3/users/swift` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:44.483397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa234e-4160-48a6-93da-06549de8988f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:44.484840 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa234e-4160-48a6-93da-06549de8988f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-60aa234e-4160-48a6-93da-06549de8988f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:44.496105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:44.497080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:44.498103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:44.498103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:44.498103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:44.498103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:44.498103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:44.498103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:44.498103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:44.498103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:44.498103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:37:44.498103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:44.498103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 300/614] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:37:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:44.504492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf19809-b8fa-4349-a607-57afe6676ee5 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:44.618829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf19809-b8fa-4349-a607-57afe6676ee5 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:44.621435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:44.655816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee92b4-7529-4c2a-a3b3-ddff0f6557f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:44.656803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee92b4-7529-4c2a-a3b3-ddff0f6557f8 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:44.657282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee92b4-7529-4c2a-a3b3-ddff0f6557f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:44.657773 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee92b4-7529-4c2a-a3b3-ddff0f6557f8 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:44.658724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee92b4-7529-4c2a-a3b3-ddff0f6557f8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:44.659248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee92b4-7529-4c2a-a3b3-ddff0f6557f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:44.660860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee92b4-7529-4c2a-a3b3-ddff0f6557f8 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:44.681087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 315/615] 10.222.0.46 () {56 vars in 1135 bytes} [Fri Sep 9 12:37:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:44.687189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa234e-4160-48a6-93da-06549de8988f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:44.814584 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa234e-4160-48a6-93da-06549de8988f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:44.859697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be619920-45de-487d-bf95-e1090542aec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:44.860329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be619920-45de-487d-bf95-e1090542aec0 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:44.860611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be619920-45de-487d-bf95-e1090542aec0 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:44.860908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be619920-45de-487d-bf95-e1090542aec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:44.878061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be619920-45de-487d-bf95-e1090542aec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:44.879547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be619920-45de-487d-bf95-e1090542aec0 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-be619920-45de-487d-bf95-e1090542aec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:44.898529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:44.899689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:44.900595 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:44.900595 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:44.900595 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:44.900595 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 301/616] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:44.906761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee92b4-7529-4c2a-a3b3-ddff0f6557f8 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:44.928397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c96b4-9c66-4f2f-9894-ae6d8bfdfcb1 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:44.928672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c96b4-9c66-4f2f-9894-ae6d8bfdfcb1 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:44.929623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c96b4-9c66-4f2f-9894-ae6d8bfdfcb1 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:44.931747 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 302/617] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:44.938511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c542d28-95c9-4245-9160-9f4bd5e0e332 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:44.938511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c542d28-95c9-4245-9160-9f4bd5e0e332 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:44.938832 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c542d28-95c9-4245-9160-9f4bd5e0e332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:44.979518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-3c542d28-95c9-4245-9160-9f4bd5e0e332 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:44.980732 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3c542d28-95c9-4245-9160-9f4bd5e0e332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:45.045453 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee92b4-7529-4c2a-a3b3-ddff0f6557f8 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:45.083456 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73acf0-f44e-4374-82af-1c816b2327c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:45.084391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73acf0-f44e-4374-82af-1c816b2327c8 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:45.084921 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73acf0-f44e-4374-82af-1c816b2327c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:45.086056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73acf0-f44e-4374-82af-1c816b2327c8 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:45.087167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73acf0-f44e-4374-82af-1c816b2327c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:45.087883 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73acf0-f44e-4374-82af-1c816b2327c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:45.089428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73acf0-f44e-4374-82af-1c816b2327c8 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:45.096313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:45.102389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 316/618] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:45.108013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73acf0-f44e-4374-82af-1c816b2327c8 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:45.142900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:45.229055 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73acf0-f44e-4374-82af-1c816b2327c8 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:45.269866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff0dab-75c6-4b03-88f7-891702948f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:45.270557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff0dab-75c6-4b03-88f7-891702948f09 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:45.270734 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 303/619] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:37:45.270846 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff0dab-75c6-4b03-88f7-891702948f09 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:45.271146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff0dab-75c6-4b03-88f7-891702948f09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:45.279326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff0dab-75c6-4b03-88f7-891702948f09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:45.280752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff0dab-75c6-4b03-88f7-891702948f09 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-88ff0dab-75c6-4b03-88f7-891702948f09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:45.292713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:45.293645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:45.294566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:45.294566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:45.294566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:45.294566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:45.294566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:45.294566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:45.294566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:45.294566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:45.294566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 317/620] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:45.299730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c542d28-95c9-4245-9160-9f4bd5e0e332 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:45.365728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc2871-171b-479d-852e-bd5cafb30cb1 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:45.365903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc2871-171b-479d-852e-bd5cafb30cb1 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:45.366175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc2871-171b-479d-852e-bd5cafb30cb1 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:45.370835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 318/621] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:37:45.383943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe25485-5cfe-4ac5-b69d-80d769fe0a7e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:45.384488 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe25485-5cfe-4ac5-b69d-80d769fe0a7e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:45.384955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe25485-5cfe-4ac5-b69d-80d769fe0a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:45.424338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-1fe25485-5cfe-4ac5-b69d-80d769fe0a7e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:45.425518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1fe25485-5cfe-4ac5-b69d-80d769fe0a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:45.442830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c542d28-95c9-4245-9160-9f4bd5e0e332 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:45.483637 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b93a55-1aa1-4c89-bc1d-0ede40234663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:45.485158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b93a55-1aa1-4c89-bc1d-0ede40234663 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:45.485794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b93a55-1aa1-4c89-bc1d-0ede40234663 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:45.486847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b93a55-1aa1-4c89-bc1d-0ede40234663 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:45.488259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b93a55-1aa1-4c89-bc1d-0ede40234663 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:45.489284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b93a55-1aa1-4c89-bc1d-0ede40234663 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:45.491338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b93a55-1aa1-4c89-bc1d-0ede40234663 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:45.516829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 304/622] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:45.527461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b93a55-1aa1-4c89-bc1d-0ede40234663 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:45.545267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 319/623] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 09 12:37:45.574527 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:45.590537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe25485-5cfe-4ac5-b69d-80d769fe0a7e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:45.618920 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:45.722885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b93a55-1aa1-4c89-bc1d-0ede40234663 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:45.756105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe25485-5cfe-4ac5-b69d-80d769fe0a7e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:45.802171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639bda1-da10-4d7f-b95e-d4db2a493f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:45.802834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639bda1-da10-4d7f-b95e-d4db2a493f2b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:45.803113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639bda1-da10-4d7f-b95e-d4db2a493f2b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:45.803381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639bda1-da10-4d7f-b95e-d4db2a493f2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:45.805132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bd03d-0ab7-4af3-9b2a-34c303e941a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:45.805786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bd03d-0ab7-4af3-9b2a-34c303e941a3 None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:45.806057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bd03d-0ab7-4af3-9b2a-34c303e941a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:45.806320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bd03d-0ab7-4af3-9b2a-34c303e941a3 None admin] PATH_INFO: `/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/e8e4805f0b1e484c98fb399313f88ef2/roles/4a5e4b92c86041c6aa22c4f4735b4a05` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:45.825451 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639bda1-da10-4d7f-b95e-d4db2a493f2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-b639bda1-da10-4d7f-b95e-d4db2a493f2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:45.830191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:45.831215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:45.832090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:45.832090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:45.832090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:45.832090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:45.832090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:45.832090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:45.832090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:45.832090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:45.832090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:45.832090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:45.832090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:45.832090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:45.837625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 320/624] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:45.841555 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85bd03d-0ab7-4af3-9b2a-34c303e941a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e2dc716277d42ae8316ebb2d36293a8, user_id=e8e4805f0b1e484c98fb399313f88ef2, role_id=4a5e4b92c86041c6aa22c4f4735b4a05)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:45.843033 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85bd03d-0ab7-4af3-9b2a-34c303e941a3 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:45.845482 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639bda1-da10-4d7f-b95e-d4db2a493f2b None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:45.919617 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:45.936412 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 305/625] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:37:45 2022] PUT /identity/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/e8e4805f0b1e484c98fb399313f88ef2/roles/4a5e4b92c86041c6aa22c4f4735b4a05 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:37:45.937262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bd03d-0ab7-4af3-9b2a-34c303e941a3 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:45.961196 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639bda1-da10-4d7f-b95e-d4db2a493f2b None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:46.007749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c58743-867d-421b-a55a-0078955a9245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:46.008938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c58743-867d-421b-a55a-0078955a9245 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:46.009602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c58743-867d-421b-a55a-0078955a9245 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:46.010098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c58743-867d-421b-a55a-0078955a9245 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:46.011334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c58743-867d-421b-a55a-0078955a9245 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:46.012269 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c58743-867d-421b-a55a-0078955a9245 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:46.013935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c58743-867d-421b-a55a-0078955a9245 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:46.026115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 321/626] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:46.035065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c58743-867d-421b-a55a-0078955a9245 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:46.096912 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:46.129229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bd03d-0ab7-4af3-9b2a-34c303e941a3 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:46.172406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac34a78-3407-4e24-a27f-84eb42feb8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:46.172998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac34a78-3407-4e24-a27f-84eb42feb8a4 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:46.173330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac34a78-3407-4e24-a27f-84eb42feb8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:46.173650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac34a78-3407-4e24-a27f-84eb42feb8a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:46.187876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac34a78-3407-4e24-a27f-84eb42feb8a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-fac34a78-3407-4e24-a27f-84eb42feb8a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:46.192484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:46.193430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:46.194267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:46.194267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:46.194267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:46.194267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:46.194267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:46.194267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:46.194267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:46.194267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:46.194267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:46.194267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:46.194267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:46.194267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:46.196478 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 306/627] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:46.202517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac34a78-3407-4e24-a27f-84eb42feb8a4 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:46.285351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c58743-867d-421b-a55a-0078955a9245 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:46.350347 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becde45c-c454-4622-ab47-ad886b084371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:46.352997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becde45c-c454-4622-ab47-ad886b084371 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:46.353502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becde45c-c454-4622-ab47-ad886b084371 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:46.353981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becde45c-c454-4622-ab47-ad886b084371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:46.357769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:46.364421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac34a78-3407-4e24-a27f-84eb42feb8a4 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:46.380006 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becde45c-c454-4622-ab47-ad886b084371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:46.381865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becde45c-c454-4622-ab47-ad886b084371 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-becde45c-c454-4622-ab47-ad886b084371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:46.413125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:46.415782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:46.417641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:46.417641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:46.417641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:46.433412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 322/628] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:37:46.437633 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becde45c-c454-4622-ab47-ad886b084371 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:46.476714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007d7d2-dc1e-4202-911b-01431d71b5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:46.477335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007d7d2-dc1e-4202-911b-01431d71b5ff None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:46.477611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007d7d2-dc1e-4202-911b-01431d71b5ff None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:46.477879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007d7d2-dc1e-4202-911b-01431d71b5ff None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:46.478565 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007d7d2-dc1e-4202-911b-01431d71b5ff None admin] RBAC: Adding query filter params (name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:46.478862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007d7d2-dc1e-4202-911b-01431d71b5ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:46.480228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007d7d2-dc1e-4202-911b-01431d71b5ff None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:46.481769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:46.501694 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 307/629] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:46.503291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e155ee3-b8d4-4c88-83ee-68b252288f11 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:46.503784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e155ee3-b8d4-4c88-83ee-68b252288f11 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:46.504233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e155ee3-b8d4-4c88-83ee-68b252288f11 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:46.505869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 308/630] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:46.510564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e155ee3-b8d4-4c88-83ee-68b252288f11 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:46.561726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becde45c-c454-4622-ab47-ad886b084371 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:46.595652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c044ce-c322-4003-8e17-bff86d53d08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:46.596368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c044ce-c322-4003-8e17-bff86d53d08f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:46.596651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c044ce-c322-4003-8e17-bff86d53d08f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:46.596937 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c044ce-c322-4003-8e17-bff86d53d08f None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:46.597592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c044ce-c322-4003-8e17-bff86d53d08f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:46.597887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c044ce-c322-4003-8e17-bff86d53d08f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:46.599268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c044ce-c322-4003-8e17-bff86d53d08f None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:46.617195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 323/631] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:46.617195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833b243-c768-4eab-a608-4fdc253760d7 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:46.617195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833b243-c768-4eab-a608-4fdc253760d7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:46.617195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833b243-c768-4eab-a608-4fdc253760d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:46.634611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0833b243-c768-4eab-a608-4fdc253760d7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:46.635520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e155ee3-b8d4-4c88-83ee-68b252288f11 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:46.635797 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0833b243-c768-4eab-a608-4fdc253760d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:46.671038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06a036-7e2c-4ddd-a622-ae52cbc505b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:46.671669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06a036-7e2c-4ddd-a622-ae52cbc505b6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:46.671947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06a036-7e2c-4ddd-a622-ae52cbc505b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:46.672223 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06a036-7e2c-4ddd-a622-ae52cbc505b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:46.682113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:46.683206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06a036-7e2c-4ddd-a622-ae52cbc505b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:46.684759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06a036-7e2c-4ddd-a622-ae52cbc505b6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-7c06a036-7e2c-4ddd-a622-ae52cbc505b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:46.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:46.700869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:46.701824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:46.701824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:46.701824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:46.701824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 309/632] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:46.702244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3c3f6-e14c-4ec6-8879-4c1addcfbbd9 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:46.702526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3c3f6-e14c-4ec6-8879-4c1addcfbbd9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:46.702805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3c3f6-e14c-4ec6-8879-4c1addcfbbd9 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:46.706521 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 310/633] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:46.706521 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3c3f6-e14c-4ec6-8879-4c1addcfbbd9 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:46.712872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 324/634] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:46.713949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833b243-c768-4eab-a608-4fdc253760d7 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:46.784197 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:46.825575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833b243-c768-4eab-a608-4fdc253760d7 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:46.858845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3c3f6-e14c-4ec6-8879-4c1addcfbbd9 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:46.873775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c98d3d-31a2-4c61-a2b6-760ca2fb8986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:46.875034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c98d3d-31a2-4c61-a2b6-760ca2fb8986 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:46.875415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c98d3d-31a2-4c61-a2b6-760ca2fb8986 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:46.875787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c98d3d-31a2-4c61-a2b6-760ca2fb8986 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:46.876788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c98d3d-31a2-4c61-a2b6-760ca2fb8986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:46.877159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c98d3d-31a2-4c61-a2b6-760ca2fb8986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:46.879282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c98d3d-31a2-4c61-a2b6-760ca2fb8986 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:46.894535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 325/635] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:46.895802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9e0fa-f715-452f-b547-f377fcd29ae7 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:46.896157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9e0fa-f715-452f-b547-f377fcd29ae7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:46.896506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9e0fa-f715-452f-b547-f377fcd29ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:46.897650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d445e0-fc79-4d29-9487-1253d9400684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:46.899389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d445e0-fc79-4d29-9487-1253d9400684 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:46.899854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d445e0-fc79-4d29-9487-1253d9400684 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:46.900317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d445e0-fc79-4d29-9487-1253d9400684 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:46.909686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d445e0-fc79-4d29-9487-1253d9400684 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:46.911478 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d445e0-fc79-4d29-9487-1253d9400684 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:46.918612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b0b9e0fa-f715-452f-b547-f377fcd29ae7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:46.920114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b0b9e0fa-f715-452f-b547-f377fcd29ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-d9d445e0-fc79-4d29-9487-1253d9400684 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:46.925413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:46.926384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:46.927382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:46.927382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:46.927382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:46.927382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:46.927382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:46.927382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:46.927382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:46.927382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:46.927382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:37:46.927382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:46.935208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 311/636] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:46.936390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d445e0-fc79-4d29-9487-1253d9400684 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:46.957600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:47.012758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 326/637] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:47.013964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9e0fa-f715-452f-b547-f377fcd29ae7 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:47.069031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d445e0-fc79-4d29-9487-1253d9400684 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:47.078600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:47.114627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104af3e-90b5-4c4e-a1ca-6fc8b59c92c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:47.115277 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104af3e-90b5-4c4e-a1ca-6fc8b59c92c6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:47.115531 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104af3e-90b5-4c4e-a1ca-6fc8b59c92c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:47.115817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104af3e-90b5-4c4e-a1ca-6fc8b59c92c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:47.125760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104af3e-90b5-4c4e-a1ca-6fc8b59c92c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:47.127286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104af3e-90b5-4c4e-a1ca-6fc8b59c92c6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:47.136802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9e0fa-f715-452f-b547-f377fcd29ae7 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-a104af3e-90b5-4c4e-a1ca-6fc8b59c92c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:47.142364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:47.143346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:47.144298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:47.144298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:47.144298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:47.144298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:47.144298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:47.144298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:47.144298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:47.144298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:47.144298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:37:47.144298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:47.144298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 312/638] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 12:37:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:47.144907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104af3e-90b5-4c4e-a1ca-6fc8b59c92c6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:47.177854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f114f9c-131b-46da-81ae-e84b06fbf186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:47.179013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f114f9c-131b-46da-81ae-e84b06fbf186 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:47.179663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f114f9c-131b-46da-81ae-e84b06fbf186 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:47.180184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f114f9c-131b-46da-81ae-e84b06fbf186 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:47.180919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f114f9c-131b-46da-81ae-e84b06fbf186 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:47.181224 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f114f9c-131b-46da-81ae-e84b06fbf186 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:47.182652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f114f9c-131b-46da-81ae-e84b06fbf186 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:47.203908 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 327/639] 10.222.0.46 () {56 vars in 1137 bytes} [Fri Sep 9 12:37:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:47.204980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f114f9c-131b-46da-81ae-e84b06fbf186 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:47.233346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:47.273642 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104af3e-90b5-4c4e-a1ca-6fc8b59c92c6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:47.315185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f114f9c-131b-46da-81ae-e84b06fbf186 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:47.317408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b5b8e-d8d9-4dbb-b7cc-133e3c6114f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:47.318712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b5b8e-d8d9-4dbb-b7cc-133e3c6114f3 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:47.322851 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b5b8e-d8d9-4dbb-b7cc-133e3c6114f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:47.323240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b5b8e-d8d9-4dbb-b7cc-133e3c6114f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:47.337796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b5b8e-d8d9-4dbb-b7cc-133e3c6114f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-744b5b8e-d8d9-4dbb-b7cc-133e3c6114f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:47.344837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:47.345807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:47.346682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:47.346682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:47.346682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:47.346682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:47.346682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:47.346682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:47.346682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:47.346682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:47.346682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:47.346682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:47.346682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:47.346682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:47.347327 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 313/640] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:47.347955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b5b8e-d8d9-4dbb-b7cc-133e3c6114f3 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:47.355727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452e5b2-0a20-4444-a93e-37bab187e3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:47.357002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452e5b2-0a20-4444-a93e-37bab187e3a9 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:47.357479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452e5b2-0a20-4444-a93e-37bab187e3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:47.357929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452e5b2-0a20-4444-a93e-37bab187e3a9 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:47.359245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452e5b2-0a20-4444-a93e-37bab187e3a9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:47.359781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452e5b2-0a20-4444-a93e-37bab187e3a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:47.361942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452e5b2-0a20-4444-a93e-37bab187e3a9 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:47.378222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:47.384846 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 328/641] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 12:37:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:47.385906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452e5b2-0a20-4444-a93e-37bab187e3a9 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:47.458308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b5b8e-d8d9-4dbb-b7cc-133e3c6114f3 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:47.474253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:47.493448 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512e8d5-8100-49fd-aa27-2c4de54085a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:47.496844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452e5b2-0a20-4444-a93e-37bab187e3a9 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:47.496954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512e8d5-8100-49fd-aa27-2c4de54085a5 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:47.496954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512e8d5-8100-49fd-aa27-2c4de54085a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:47.496954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512e8d5-8100-49fd-aa27-2c4de54085a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:47.530059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d19923-4097-498e-bf19-5a798f72711c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:47.531012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d19923-4097-498e-bf19-5a798f72711c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:47.531430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d19923-4097-498e-bf19-5a798f72711c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:47.531862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d19923-4097-498e-bf19-5a798f72711c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:47.543184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d19923-4097-498e-bf19-5a798f72711c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-30d19923-4097-498e-bf19-5a798f72711c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:47.548271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:47.549454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:47.550532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:47.550532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:47.550532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:47.550532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:47.550532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:47.550532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:47.550532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:47.550532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:47.550532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:47.550532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:47.550532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:47.550532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:47.552640 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 329/642] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:47.553672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d19923-4097-498e-bf19-5a798f72711c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:47.568869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512e8d5-8100-49fd-aa27-2c4de54085a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:47.570596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512e8d5-8100-49fd-aa27-2c4de54085a5 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:47.597111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:47.658294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d19923-4097-498e-bf19-5a798f72711c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:47.671241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 314/643] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:47.671953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512e8d5-8100-49fd-aa27-2c4de54085a5 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:47.699996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd22c98-e319-45f4-bd01-7f1a3fea1919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:47.700979 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd22c98-e319-45f4-bd01-7f1a3fea1919 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:47.701503 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd22c98-e319-45f4-bd01-7f1a3fea1919 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:47.701992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd22c98-e319-45f4-bd01-7f1a3fea1919 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:47.702871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd22c98-e319-45f4-bd01-7f1a3fea1919 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:47.703365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd22c98-e319-45f4-bd01-7f1a3fea1919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:47.705099 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd22c98-e319-45f4-bd01-7f1a3fea1919 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:47.719672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 330/644] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:47.720555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd22c98-e319-45f4-bd01-7f1a3fea1919 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:47.724255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:47.774057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512e8d5-8100-49fd-aa27-2c4de54085a5 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:47.801251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:47.809842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a7428-29c7-44d2-9009-c8f3aa45af21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:47.810733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a7428-29c7-44d2-9009-c8f3aa45af21 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:47.811016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a7428-29c7-44d2-9009-c8f3aa45af21 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:47.811288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a7428-29c7-44d2-9009-c8f3aa45af21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:47.829695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd22c98-e319-45f4-bd01-7f1a3fea1919 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:47.864292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9429bf2-f970-4a9c-85be-889ab1a43d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:47.865158 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9429bf2-f970-4a9c-85be-889ab1a43d00 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:47.865594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9429bf2-f970-4a9c-85be-889ab1a43d00 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:47.866114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9429bf2-f970-4a9c-85be-889ab1a43d00 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:47.867593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9429bf2-f970-4a9c-85be-889ab1a43d00 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:47.868049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9429bf2-f970-4a9c-85be-889ab1a43d00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:47.869537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9429bf2-f970-4a9c-85be-889ab1a43d00 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:47.878328 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 331/645] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:47.879513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9429bf2-f970-4a9c-85be-889ab1a43d00 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:47.890514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a7428-29c7-44d2-9009-c8f3aa45af21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:47.892161 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a7428-29c7-44d2-9009-c8f3aa45af21 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:47.946835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:47.982475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9429bf2-f970-4a9c-85be-889ab1a43d00 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:48.007760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 315/646] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:48.008960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a7428-29c7-44d2-9009-c8f3aa45af21 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:48.023370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d4d57-57a0-400e-8e80-993d52e77c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:48.024107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d4d57-57a0-400e-8e80-993d52e77c97 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:48.024383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d4d57-57a0-400e-8e80-993d52e77c97 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:48.024654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d4d57-57a0-400e-8e80-993d52e77c97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:48.035703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d4d57-57a0-400e-8e80-993d52e77c97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:48.037116 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d4d57-57a0-400e-8e80-993d52e77c97 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:48.040209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-502d4d57-57a0-400e-8e80-993d52e77c97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:48.052125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:48.053390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:48.055869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:48.055869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:48.055869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:48.055869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 332/647] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:48.055869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d4d57-57a0-400e-8e80-993d52e77c97 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:48.129355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a7428-29c7-44d2-9009-c8f3aa45af21 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:48.166930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c5121-60fe-4417-b5a9-3437ad4c0c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:48.167586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c5121-60fe-4417-b5a9-3437ad4c0c1c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:48.167860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c5121-60fe-4417-b5a9-3437ad4c0c1c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:48.168131 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c5121-60fe-4417-b5a9-3437ad4c0c1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:48.168297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d4d57-57a0-400e-8e80-993d52e77c97 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:48.177959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c5121-60fe-4417-b5a9-3437ad4c0c1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:48.180023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c5121-60fe-4417-b5a9-3437ad4c0c1c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:48.188245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-002c5121-60fe-4417-b5a9-3437ad4c0c1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:48.195112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:48.201324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:48.202442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:48.202442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:48.202442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:48.202442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 316/648] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:48.202442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c5121-60fe-4417-b5a9-3437ad4c0c1c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:48.213340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861c1b5-8b12-442f-9817-82e3a36eb958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:48.214002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861c1b5-8b12-442f-9817-82e3a36eb958 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:48.214274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861c1b5-8b12-442f-9817-82e3a36eb958 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:48.214570 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861c1b5-8b12-442f-9817-82e3a36eb958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:48.228808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861c1b5-8b12-442f-9817-82e3a36eb958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:48.230405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861c1b5-8b12-442f-9817-82e3a36eb958 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-2861c1b5-8b12-442f-9817-82e3a36eb958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:48.248161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:48.253812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:48.254951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:48.254951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:48.254951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:48.254951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 333/649] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:48.254951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861c1b5-8b12-442f-9817-82e3a36eb958 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:48.308803 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:48.329048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c5121-60fe-4417-b5a9-3437ad4c0c1c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:48.363160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861c1b5-8b12-442f-9817-82e3a36eb958 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:48.363679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9279ae4-fbd8-45ff-a933-c1c4e08edca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:48.364326 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9279ae4-fbd8-45ff-a933-c1c4e08edca9 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:48.364604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9279ae4-fbd8-45ff-a933-c1c4e08edca9 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:48.364872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9279ae4-fbd8-45ff-a933-c1c4e08edca9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:48.375194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9279ae4-fbd8-45ff-a933-c1c4e08edca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:48.376634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9279ae4-fbd8-45ff-a933-c1c4e08edca9 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-c9279ae4-fbd8-45ff-a933-c1c4e08edca9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:48.390240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:48.391365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:48.392287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:48.392287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:48.392287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:48.392287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 317/650] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:48.392287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9279ae4-fbd8-45ff-a933-c1c4e08edca9 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:48.399275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf07a99-d5d6-4fa2-80a9-8fa40eb98273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:48.399882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf07a99-d5d6-4fa2-80a9-8fa40eb98273 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:48.400160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf07a99-d5d6-4fa2-80a9-8fa40eb98273 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:48.400431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf07a99-d5d6-4fa2-80a9-8fa40eb98273 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:48.401075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf07a99-d5d6-4fa2-80a9-8fa40eb98273 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:48.401375 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf07a99-d5d6-4fa2-80a9-8fa40eb98273 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:48.402756 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf07a99-d5d6-4fa2-80a9-8fa40eb98273 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:48.414873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 334/651] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:48.415664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf07a99-d5d6-4fa2-80a9-8fa40eb98273 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:48.447880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:48.504955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9279ae4-fbd8-45ff-a933-c1c4e08edca9 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:48.516898 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf07a99-d5d6-4fa2-80a9-8fa40eb98273 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:48.533507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:48.538315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a839d39-1ebd-497d-b587-bf23fbd18f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:48.539330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a839d39-1ebd-497d-b587-bf23fbd18f2f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:48.539619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a839d39-1ebd-497d-b587-bf23fbd18f2f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:48.539899 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a839d39-1ebd-497d-b587-bf23fbd18f2f None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:48.540529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a839d39-1ebd-497d-b587-bf23fbd18f2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:48.540842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a839d39-1ebd-497d-b587-bf23fbd18f2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:48.542195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a839d39-1ebd-497d-b587-bf23fbd18f2f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:48.549923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297774c3-17fb-486b-8484-169b1f1250b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:48.550549 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297774c3-17fb-486b-8484-169b1f1250b1 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:48.550832 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297774c3-17fb-486b-8484-169b1f1250b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:48.551102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297774c3-17fb-486b-8484-169b1f1250b1 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:48.551774 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297774c3-17fb-486b-8484-169b1f1250b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:48.552075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297774c3-17fb-486b-8484-169b1f1250b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:48.553457 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297774c3-17fb-486b-8484-169b1f1250b1 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:48.554699 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 318/652] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:48.556159 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c41bf-140f-4329-9e8a-7fb10297acf5 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:48.556444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c41bf-140f-4329-9e8a-7fb10297acf5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:48.556715 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c41bf-140f-4329-9e8a-7fb10297acf5 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:48.558072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 319/653] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:48.558868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c41bf-140f-4329-9e8a-7fb10297acf5 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:48.572737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 335/654] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:48.573558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297774c3-17fb-486b-8484-169b1f1250b1 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:48.660195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:48.661904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c41bf-140f-4329-9e8a-7fb10297acf5 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:48.670568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297774c3-17fb-486b-8484-169b1f1250b1 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:48.696541 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19ff7a-2292-482f-a521-2f0b18629c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:48.697140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19ff7a-2292-482f-a521-2f0b18629c0a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:48.697417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19ff7a-2292-482f-a521-2f0b18629c0a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:48.697687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19ff7a-2292-482f-a521-2f0b18629c0a None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:48.698312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19ff7a-2292-482f-a521-2f0b18629c0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:48.698646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19ff7a-2292-482f-a521-2f0b18629c0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:48.700007 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19ff7a-2292-482f-a521-2f0b18629c0a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:48.704117 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3ba3-fca6-4b9a-b4c7-f1906df31fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:48.704736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3ba3-fca6-4b9a-b4c7-f1906df31fb2 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:48.705009 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3ba3-fca6-4b9a-b4c7-f1906df31fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:48.705277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3ba3-fca6-4b9a-b4c7-f1906df31fb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:48.711216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 320/655] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:48.712008 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19ff7a-2292-482f-a521-2f0b18629c0a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:48.712501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3ba3-fca6-4b9a-b4c7-f1906df31fb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:48.713966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3ba3-fca6-4b9a-b4c7-f1906df31fb2 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-1d8f3ba3-fca6-4b9a-b4c7-f1906df31fb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:48.726281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:48.727357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:48.728243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:48.728243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:48.728243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:48.728243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:48.728243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:48.728243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:48.728243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:48.728243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:48.728243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 336/656] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:48.729625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99100f4b-7810-40a6-95ac-7497acbe6ccb None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:48.729908 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99100f4b-7810-40a6-95ac-7497acbe6ccb None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:48.730178 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99100f4b-7810-40a6-95ac-7497acbe6ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:48.748315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-99100f4b-7810-40a6-95ac-7497acbe6ccb None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:48.749526 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-99100f4b-7810-40a6-95ac-7497acbe6ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:48.771133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:48.818020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 337/657] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:48.819134 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99100f4b-7810-40a6-95ac-7497acbe6ccb None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:48.823577 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19ff7a-2292-482f-a521-2f0b18629c0a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:48.858915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4187e2-5bc9-4136-bf8d-47fde2a94961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:48.859946 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4187e2-5bc9-4136-bf8d-47fde2a94961 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:48.860227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4187e2-5bc9-4136-bf8d-47fde2a94961 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:48.860498 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4187e2-5bc9-4136-bf8d-47fde2a94961 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:48.869868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4187e2-5bc9-4136-bf8d-47fde2a94961 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:48.871337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4187e2-5bc9-4136-bf8d-47fde2a94961 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-ce4187e2-5bc9-4136-bf8d-47fde2a94961 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:48.883010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:48.883963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:48.885628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:48.885628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:48.885628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:48.885628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:48.885628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:48.885628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:48.885628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:48.885628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:48.885628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 12:37:48.885628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:48.885628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 321/658] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:48.885628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4187e2-5bc9-4136-bf8d-47fde2a94961 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:48.907255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:48.931553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99100f4b-7810-40a6-95ac-7497acbe6ccb None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:48.968486 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826192a5-299f-488f-a0a5-6eb28f3a9e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:48.969146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826192a5-299f-488f-a0a5-6eb28f3a9e26 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:48.969426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826192a5-299f-488f-a0a5-6eb28f3a9e26 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:48.969705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826192a5-299f-488f-a0a5-6eb28f3a9e26 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:48.977214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826192a5-299f-488f-a0a5-6eb28f3a9e26 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:48.978668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826192a5-299f-488f-a0a5-6eb28f3a9e26 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:48.988216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4187e2-5bc9-4136-bf8d-47fde2a94961 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-826192a5-299f-488f-a0a5-6eb28f3a9e26 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:48.990028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:48.991007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:48.992101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:48.992101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:48.992101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:48.992101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:48.992101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:48.992101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:48.992101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:48.992101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:48.992101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 12:37:48.992101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:48.992101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 338/659] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:48.993162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826192a5-299f-488f-a0a5-6eb28f3a9e26 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:49.007447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:49.030171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d167a-32ab-485c-a55a-5fec0edf41e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:49.030763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d167a-32ab-485c-a55a-5fec0edf41e0 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:49.031051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d167a-32ab-485c-a55a-5fec0edf41e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:49.031325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d167a-32ab-485c-a55a-5fec0edf41e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:49.038795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d167a-32ab-485c-a55a-5fec0edf41e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:49.040250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d167a-32ab-485c-a55a-5fec0edf41e0 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-172d167a-32ab-485c-a55a-5fec0edf41e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:49.052180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:49.053250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:49.054933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:49.054933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:49.054933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:49.054933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:49.054933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:49.054933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:49.054933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:49.054933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:49.054933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 322/660] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:49.054933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d167a-32ab-485c-a55a-5fec0edf41e0 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:49.104711 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826192a5-299f-488f-a0a5-6eb28f3a9e26 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:49.147757 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881e3d7-b1bb-4f96-959b-ddf8ff6a7382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:49.148642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881e3d7-b1bb-4f96-959b-ddf8ff6a7382 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:49.148919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881e3d7-b1bb-4f96-959b-ddf8ff6a7382 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:49.149192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881e3d7-b1bb-4f96-959b-ddf8ff6a7382 None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:49.149880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881e3d7-b1bb-4f96-959b-ddf8ff6a7382 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:49.150188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881e3d7-b1bb-4f96-959b-ddf8ff6a7382 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:49.151784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881e3d7-b1bb-4f96-959b-ddf8ff6a7382 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:49.158557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d167a-32ab-485c-a55a-5fec0edf41e0 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:49.160511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:49.164295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 339/661] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:49.165085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881e3d7-b1bb-4f96-959b-ddf8ff6a7382 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:49.193959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802bd7e-3b49-4764-8887-55ccf30e55f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:49.194654 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802bd7e-3b49-4764-8887-55ccf30e55f4 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:49.195441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802bd7e-3b49-4764-8887-55ccf30e55f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:49.195906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802bd7e-3b49-4764-8887-55ccf30e55f4 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:49.196630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802bd7e-3b49-4764-8887-55ccf30e55f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:49.196936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802bd7e-3b49-4764-8887-55ccf30e55f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:49.198380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802bd7e-3b49-4764-8887-55ccf30e55f4 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:49.216390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 323/662] 10.222.0.46 () {56 vars in 1137 bytes} [Fri Sep 9 12:37:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:49.219383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802bd7e-3b49-4764-8887-55ccf30e55f4 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:49.248452 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:49.264924 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881e3d7-b1bb-4f96-959b-ddf8ff6a7382 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:49.299499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0500c9-0725-4700-b8a4-840630ede312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:49.300025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0500c9-0725-4700-b8a4-840630ede312 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:49.300295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0500c9-0725-4700-b8a4-840630ede312 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:49.300560 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0500c9-0725-4700-b8a4-840630ede312 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:49.301272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0500c9-0725-4700-b8a4-840630ede312 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:49.301566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0500c9-0725-4700-b8a4-840630ede312 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:49.303010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0500c9-0725-4700-b8a4-840630ede312 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:49.317660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802bd7e-3b49-4764-8887-55ccf30e55f4 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:49.319490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 340/663] 10.222.0.46 () {56 vars in 1139 bytes} [Fri Sep 9 12:37:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:49.320410 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0500c9-0725-4700-b8a4-840630ede312 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:49.352072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3e6ba-c865-4cdb-a12c-de3a69c84226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:49.352667 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3e6ba-c865-4cdb-a12c-de3a69c84226 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:49.352942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3e6ba-c865-4cdb-a12c-de3a69c84226 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:49.353211 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3e6ba-c865-4cdb-a12c-de3a69c84226 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:49.353883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3e6ba-c865-4cdb-a12c-de3a69c84226 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:49.354182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3e6ba-c865-4cdb-a12c-de3a69c84226 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:49.355650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3e6ba-c865-4cdb-a12c-de3a69c84226 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:49.367545 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 324/664] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:49.368460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3e6ba-c865-4cdb-a12c-de3a69c84226 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:49.379018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:49.419769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0500c9-0725-4700-b8a4-840630ede312 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:49.452687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b912af-d5d7-4c67-aa50-9e389b7ccceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:49.453310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b912af-d5d7-4c67-aa50-9e389b7ccceb None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:49.453596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b912af-d5d7-4c67-aa50-9e389b7ccceb None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:49.453868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b912af-d5d7-4c67-aa50-9e389b7ccceb None admin] PATH_INFO: `/v3/roles/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:49.456113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:49.464269 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b912af-d5d7-4c67-aa50-9e389b7ccceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:49.467724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3e6ba-c865-4cdb-a12c-de3a69c84226 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-94b912af-d5d7-4c67-aa50-9e389b7ccceb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:49.469013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:49.469949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:49.470836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:49.470836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:49.470836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:49.470836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:49.470836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:49.470836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:49.470836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:49.470836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:49.470836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:49.470836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:49.470836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:49.470836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:49.470836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 341/665] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:49.471536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b912af-d5d7-4c67-aa50-9e389b7ccceb None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:49.499768 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018ce65-70be-4de2-a01c-2aef2823e113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:49.500372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018ce65-70be-4de2-a01c-2aef2823e113 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:49.500650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018ce65-70be-4de2-a01c-2aef2823e113 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:49.500922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018ce65-70be-4de2-a01c-2aef2823e113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:49.501631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018ce65-70be-4de2-a01c-2aef2823e113 None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=002c117a85c1452e9fae80c755c6622f) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:49.501936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018ce65-70be-4de2-a01c-2aef2823e113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:49.503384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018ce65-70be-4de2-a01c-2aef2823e113 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:49.508946 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 325/666] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:49 2022] GET /identity/v3/role_assignments?user.id=002c117a85c1452e9fae80c755c6622f&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:49.509710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018ce65-70be-4de2-a01c-2aef2823e113 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:49.583970 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b912af-d5d7-4c67-aa50-9e389b7ccceb None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:49.603756 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:49.623527 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb8cd5-56ea-477f-9cd2-dab41fec2c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:49.624149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb8cd5-56ea-477f-9cd2-dab41fec2c5e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:49.624436 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb8cd5-56ea-477f-9cd2-dab41fec2c5e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:49.624706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb8cd5-56ea-477f-9cd2-dab41fec2c5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:49.632482 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018ce65-70be-4de2-a01c-2aef2823e113 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:49.669355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba19d5c-08b8-4362-8c2c-432e9a679ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:49.669983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba19d5c-08b8-4362-8c2c-432e9a679ab5 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:49.670258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba19d5c-08b8-4362-8c2c-432e9a679ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:49.670546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba19d5c-08b8-4362-8c2c-432e9a679ab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:49.697592 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:49.704696 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb8cd5-56ea-477f-9cd2-dab41fec2c5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:49.706130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb8cd5-56ea-477f-9cd2-dab41fec2c5e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:49.752184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba19d5c-08b8-4362-8c2c-432e9a679ab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:49.753670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba19d5c-08b8-4362-8c2c-432e9a679ab5 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:49.822154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 342/667] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:49.823730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb8cd5-56ea-477f-9cd2-dab41fec2c5e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:49.844193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:49.859488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 326/668] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:49.860554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba19d5c-08b8-4362-8c2c-432e9a679ab5 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:49.917956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:49.935858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb8cd5-56ea-477f-9cd2-dab41fec2c5e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:49.964451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba19d5c-08b8-4362-8c2c-432e9a679ab5 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:49.972293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79af70-fc2c-4a2a-ae9d-b312c5c2fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:49.972893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79af70-fc2c-4a2a-ae9d-b312c5c2fd0c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:49.973177 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79af70-fc2c-4a2a-ae9d-b312c5c2fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:49.973448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79af70-fc2c-4a2a-ae9d-b312c5c2fd0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:49.974566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79af70-fc2c-4a2a-ae9d-b312c5c2fd0c None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=f29af98c5fba4569b16edf240b13c5c5) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:49.974881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79af70-fc2c-4a2a-ae9d-b312c5c2fd0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:49.977009 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79af70-fc2c-4a2a-ae9d-b312c5c2fd0c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:49.983052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 343/669] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:49 2022] GET /identity/v3/role_assignments?user.id=f29af98c5fba4569b16edf240b13c5c5&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:49.983863 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79af70-fc2c-4a2a-ae9d-b312c5c2fd0c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:50.016891 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e854c6e-24d3-476c-bb7c-c78174d37b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:50.018414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e854c6e-24d3-476c-bb7c-c78174d37b2b None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:50.019194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e854c6e-24d3-476c-bb7c-c78174d37b2b None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:50.019680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e854c6e-24d3-476c-bb7c-c78174d37b2b None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:50.021329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e854c6e-24d3-476c-bb7c-c78174d37b2b None admin] RBAC: Adding query filter params (name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:50.022013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e854c6e-24d3-476c-bb7c-c78174d37b2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:50.023821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e854c6e-24d3-476c-bb7c-c78174d37b2b None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:50.039075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 327/670] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:50.040217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e854c6e-24d3-476c-bb7c-c78174d37b2b None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:50.095683 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:50.137250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79af70-fc2c-4a2a-ae9d-b312c5c2fd0c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:50.173127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e854c6e-24d3-476c-bb7c-c78174d37b2b None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:50.181288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ffbd8-6497-474c-be01-5ad3547082ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:50.182062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ffbd8-6497-474c-be01-5ad3547082ba None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:50.182446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ffbd8-6497-474c-be01-5ad3547082ba None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:50.182903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ffbd8-6497-474c-be01-5ad3547082ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:50.195011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:50.197460 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ffbd8-6497-474c-be01-5ad3547082ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:50.198956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ffbd8-6497-474c-be01-5ad3547082ba None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:50.214948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64b077-a32c-412f-bdd7-945aa12f68a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:50.215688 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64b077-a32c-412f-bdd7-945aa12f68a7 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-f17ffbd8-6497-474c-be01-5ad3547082ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:50.215915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:50.216848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:50.217754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:50.217754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:50.217754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:50.217933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64b077-a32c-412f-bdd7-945aa12f68a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:50.217933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64b077-a32c-412f-bdd7-945aa12f68a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:50.218062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 344/671] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:50.218062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ffbd8-6497-474c-be01-5ad3547082ba None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:50.234215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff64b077-a32c-412f-bdd7-945aa12f68a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:50.235724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff64b077-a32c-412f-bdd7-945aa12f68a7 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-ff64b077-a32c-412f-bdd7-945aa12f68a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:50.253434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:50.254693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:50.255919 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:50.255919 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:50.255919 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:50.257523 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 328/672] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:50.258668 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64b077-a32c-412f-bdd7-945aa12f68a7 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:50.350323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ffbd8-6497-474c-be01-5ad3547082ba None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:50.365336 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64b077-a32c-412f-bdd7-945aa12f68a7 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:50.388781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f568aaf-f172-4421-b328-5c220a5aea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:50.389715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f568aaf-f172-4421-b328-5c220a5aea90 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:50.390095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f568aaf-f172-4421-b328-5c220a5aea90 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:50.390525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f568aaf-f172-4421-b328-5c220a5aea90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:50.392594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:50.402093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdc568-e55c-4218-9381-f05394ba1d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:50.405526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdc568-e55c-4218-9381-f05394ba1d1a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:50.405526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdc568-e55c-4218-9381-f05394ba1d1a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:50.405737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f568aaf-f172-4421-b328-5c220a5aea90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:50.405830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdc568-e55c-4218-9381-f05394ba1d1a None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:50.405830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdc568-e55c-4218-9381-f05394ba1d1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:50.405830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdc568-e55c-4218-9381-f05394ba1d1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:50.406046 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f568aaf-f172-4421-b328-5c220a5aea90 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:50.406569 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdc568-e55c-4218-9381-f05394ba1d1a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:50.419869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 329/673] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:50.420967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdc568-e55c-4218-9381-f05394ba1d1a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-0f568aaf-f172-4421-b328-5c220a5aea90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:50.421315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:50.422260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:50.423215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:50.423215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:50.423215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:50.428366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 345/674] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:50.429461 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f568aaf-f172-4421-b328-5c220a5aea90 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:50.455316 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:50.531915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdc568-e55c-4218-9381-f05394ba1d1a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:50.567091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f568aaf-f172-4421-b328-5c220a5aea90 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:50.568034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed14cf1-4ffa-4156-9f84-e303d4770fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:50.568844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed14cf1-4ffa-4156-9f84-e303d4770fc5 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:50.569318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed14cf1-4ffa-4156-9f84-e303d4770fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:50.570418 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed14cf1-4ffa-4156-9f84-e303d4770fc5 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:50.571387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed14cf1-4ffa-4156-9f84-e303d4770fc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:50.571898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed14cf1-4ffa-4156-9f84-e303d4770fc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:50.573481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed14cf1-4ffa-4156-9f84-e303d4770fc5 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:50.589469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 330/675] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:50.589469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed14cf1-4ffa-4156-9f84-e303d4770fc5 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:50.612896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79aa92f-b731-4103-a102-2faa05e3b037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:50.613981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79aa92f-b731-4103-a102-2faa05e3b037 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:50.614575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79aa92f-b731-4103-a102-2faa05e3b037 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:50.615069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79aa92f-b731-4103-a102-2faa05e3b037 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:50.623492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79aa92f-b731-4103-a102-2faa05e3b037 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:50.625197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79aa92f-b731-4103-a102-2faa05e3b037 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-e79aa92f-b731-4103-a102-2faa05e3b037 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:50.638500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:50.639428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:50.640332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:50.640332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:50.640332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:50.640332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:50.640332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:50.640332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:50.640332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:50.640332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:50.642856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 346/676] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:50.644009 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79aa92f-b731-4103-a102-2faa05e3b037 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:50.676098 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:50.689180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed14cf1-4ffa-4156-9f84-e303d4770fc5 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:50.695837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:50.724122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a88342-3fd8-48a5-978e-b0de8e96d91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:50.724735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a88342-3fd8-48a5-978e-b0de8e96d91b None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:50.725020 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a88342-3fd8-48a5-978e-b0de8e96d91b None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:50.725350 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a88342-3fd8-48a5-978e-b0de8e96d91b None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:50.726051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a88342-3fd8-48a5-978e-b0de8e96d91b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:50.726356 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a88342-3fd8-48a5-978e-b0de8e96d91b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:50.727783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a88342-3fd8-48a5-978e-b0de8e96d91b None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:50.740631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 331/677] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:50.741255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a88342-3fd8-48a5-978e-b0de8e96d91b None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:50.759642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79aa92f-b731-4103-a102-2faa05e3b037 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:50.798709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e6fd0-6802-4661-a21f-9298ad73c5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:50.799623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e6fd0-6802-4661-a21f-9298ad73c5d4 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:50.800100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e6fd0-6802-4661-a21f-9298ad73c5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:50.800474 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e6fd0-6802-4661-a21f-9298ad73c5d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:50.808718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e6fd0-6802-4661-a21f-9298ad73c5d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:50.811538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e6fd0-6802-4661-a21f-9298ad73c5d4 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-0f5e6fd0-6802-4661-a21f-9298ad73c5d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:50.824287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:50.825642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:50.827574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:50.827574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:50.827574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:50.827574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:50.827574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:50.827574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:50.827574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:50.827574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:50.827574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 347/678] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:50.828644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e6fd0-6802-4661-a21f-9298ad73c5d4 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:50.852188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a88342-3fd8-48a5-978e-b0de8e96d91b None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:50.889081 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375b9e1-b9ca-41bf-85e6-ed26e971e8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:50.889609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375b9e1-b9ca-41bf-85e6-ed26e971e8ae None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:50.889888 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375b9e1-b9ca-41bf-85e6-ed26e971e8ae None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:50.890163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375b9e1-b9ca-41bf-85e6-ed26e971e8ae None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:50.890917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0375b9e1-b9ca-41bf-85e6-ed26e971e8ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:50.891229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0375b9e1-b9ca-41bf-85e6-ed26e971e8ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:50.892669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0375b9e1-b9ca-41bf-85e6-ed26e971e8ae None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:50.905577 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 332/679] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:50.906419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375b9e1-b9ca-41bf-85e6-ed26e971e8ae None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:50.917462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:50.943835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e6fd0-6802-4661-a21f-9298ad73c5d4 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:50.951548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:50.980760 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf4518-ca16-47a7-8fde-50448d68ff9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:50.982160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf4518-ca16-47a7-8fde-50448d68ff9d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:50.982659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf4518-ca16-47a7-8fde-50448d68ff9d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:50.983103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf4518-ca16-47a7-8fde-50448d68ff9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:50.992817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf4518-ca16-47a7-8fde-50448d68ff9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:50.994519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf4518-ca16-47a7-8fde-50448d68ff9d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-daaf4518-ca16-47a7-8fde-50448d68ff9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:51.006378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:51.008042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:51.009071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:51.009071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:51.009071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:51.009071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:51.009071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:51.009071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:51.009071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:51.009071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:51.009071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:37:51.009071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:51.015702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375b9e1-b9ca-41bf-85e6-ed26e971e8ae None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:51.015907 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 348/680] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:37:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:51.016883 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf4518-ca16-47a7-8fde-50448d68ff9d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:51.046508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcfc5d-ba84-4124-8ac7-941fdeb76cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:51.047287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcfc5d-ba84-4124-8ac7-941fdeb76cbf None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:51.047673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcfc5d-ba84-4124-8ac7-941fdeb76cbf None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:51.048061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcfc5d-ba84-4124-8ac7-941fdeb76cbf None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:51.048828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bcfc5d-ba84-4124-8ac7-941fdeb76cbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:51.049238 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bcfc5d-ba84-4124-8ac7-941fdeb76cbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:51.050934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bcfc5d-ba84-4124-8ac7-941fdeb76cbf None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:51.068253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 333/681] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:51.069211 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcfc5d-ba84-4124-8ac7-941fdeb76cbf None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:51.123841 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf4518-ca16-47a7-8fde-50448d68ff9d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:51.151925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:51.161007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48174a50-4eb0-4d7d-88a5-bf001d554954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:51.161987 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48174a50-4eb0-4d7d-88a5-bf001d554954 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:51.162780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48174a50-4eb0-4d7d-88a5-bf001d554954 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:51.163205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48174a50-4eb0-4d7d-88a5-bf001d554954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:51.164474 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48174a50-4eb0-4d7d-88a5-bf001d554954 None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=eff8fcb7fa2740bbb801badf800d2d6a) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:51.165093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48174a50-4eb0-4d7d-88a5-bf001d554954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:51.167362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48174a50-4eb0-4d7d-88a5-bf001d554954 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:51.171623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcfc5d-ba84-4124-8ac7-941fdeb76cbf None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:51.175787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 349/682] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:51 2022] GET /identity/v3/role_assignments?user.id=eff8fcb7fa2740bbb801badf800d2d6a&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:51.181846 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48174a50-4eb0-4d7d-88a5-bf001d554954 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:51.232481 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:51.279839 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338e125-a2b2-4ed5-bee8-a6f786beff35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:51.280525 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338e125-a2b2-4ed5-bee8-a6f786beff35 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:51.280740 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338e125-a2b2-4ed5-bee8-a6f786beff35 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:51.281010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338e125-a2b2-4ed5-bee8-a6f786beff35 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:51.281749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338e125-a2b2-4ed5-bee8-a6f786beff35 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:51.282050 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338e125-a2b2-4ed5-bee8-a6f786beff35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:51.283539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338e125-a2b2-4ed5-bee8-a6f786beff35 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:51.317049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48174a50-4eb0-4d7d-88a5-bf001d554954 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:51.333572 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 334/683] 10.222.0.46 () {56 vars in 1135 bytes} [Fri Sep 9 12:37:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:51.338981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338e125-a2b2-4ed5-bee8-a6f786beff35 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:51.367157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326104a-a163-46ff-a07a-44c4c44defd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:51.368264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326104a-a163-46ff-a07a-44c4c44defd7 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:51.368781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326104a-a163-46ff-a07a-44c4c44defd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:51.369289 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326104a-a163-46ff-a07a-44c4c44defd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:51.370289 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326104a-a163-46ff-a07a-44c4c44defd7 None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=e46bd9d49bf64013ae58770b0284d81e) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:51.371013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326104a-a163-46ff-a07a-44c4c44defd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:51.372894 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326104a-a163-46ff-a07a-44c4c44defd7 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:51.380652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 350/684] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:51 2022] GET /identity/v3/role_assignments?user.id=e46bd9d49bf64013ae58770b0284d81e&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:51.498392 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338e125-a2b2-4ed5-bee8-a6f786beff35 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:51.536802 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c2a57-2f55-4ff8-bf7f-5da15d17fff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:51.537487 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c2a57-2f55-4ff8-bf7f-5da15d17fff2 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:51.537773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c2a57-2f55-4ff8-bf7f-5da15d17fff2 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:51.538047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c2a57-2f55-4ff8-bf7f-5da15d17fff2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:51.542884 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:51.623399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c2a57-2f55-4ff8-bf7f-5da15d17fff2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:51.624912 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c2a57-2f55-4ff8-bf7f-5da15d17fff2 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:51.735585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 335/685] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:51.741581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326104a-a163-46ff-a07a-44c4c44defd7 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:51.848384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ebf69-48b4-4f00-a97a-821c5b4cced2 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:51.848582 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ebf69-48b4-4f00-a97a-821c5b4cced2 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:51.848853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ebf69-48b4-4f00-a97a-821c5b4cced2 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:51.850194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 336/686] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:51.866233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fff51-24e1-478a-b1fe-a2b9b7bed765 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:51.867278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fff51-24e1-478a-b1fe-a2b9b7bed765 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:51.867701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fff51-24e1-478a-b1fe-a2b9b7bed765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:51.878302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:51.891952 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326104a-a163-46ff-a07a-44c4c44defd7 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:51.901317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-2f9fff51-24e1-478a-b1fe-a2b9b7bed765 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:51.903009 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2f9fff51-24e1-478a-b1fe-a2b9b7bed765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:51.937185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:51.962418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd6e2a-5224-49d1-a02c-20baecedcda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:51.963096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd6e2a-5224-49d1-a02c-20baecedcda8 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:51.963375 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd6e2a-5224-49d1-a02c-20baecedcda8 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:51.963649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd6e2a-5224-49d1-a02c-20baecedcda8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:51.995604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd6e2a-5224-49d1-a02c-20baecedcda8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:51.997060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd6e2a-5224-49d1-a02c-20baecedcda8 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:52.017266 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 337/687] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-bbbd6e2a-5224-49d1-a02c-20baecedcda8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:52.021489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:52.022434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:52.024148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:52.024148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:52.024148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:52.024148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 351/688] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:52.029970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fff51-24e1-478a-b1fe-a2b9b7bed765 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:52.163870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fff51-24e1-478a-b1fe-a2b9b7bed765 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:52.219311 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89413f9a-2075-4db1-847c-3ff8e40f6eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:52.219921 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89413f9a-2075-4db1-847c-3ff8e40f6eb7 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:52.220198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89413f9a-2075-4db1-847c-3ff8e40f6eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:52.220466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89413f9a-2075-4db1-847c-3ff8e40f6eb7 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:52.221091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89413f9a-2075-4db1-847c-3ff8e40f6eb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:52.221386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89413f9a-2075-4db1-847c-3ff8e40f6eb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:52.222827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89413f9a-2075-4db1-847c-3ff8e40f6eb7 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:52.248934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 338/689] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:52.254387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd6e2a-5224-49d1-a02c-20baecedcda8 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:52.370217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd6e2a-5224-49d1-a02c-20baecedcda8 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:52.404843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d359816-226c-43ca-b8a5-dc190197dab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:52.405391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d359816-226c-43ca-b8a5-dc190197dab2 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:52.405665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d359816-226c-43ca-b8a5-dc190197dab2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:52.405934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d359816-226c-43ca-b8a5-dc190197dab2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:52.412879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d359816-226c-43ca-b8a5-dc190197dab2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:52.415165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d359816-226c-43ca-b8a5-dc190197dab2 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-2d359816-226c-43ca-b8a5-dc190197dab2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:52.427651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:52.428612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:52.430566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:52.430566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:52.430566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:52.430566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:52.430566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:52.430566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:52.430566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:52.430566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:52.430566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 352/690] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:52.437505 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89413f9a-2075-4db1-847c-3ff8e40f6eb7 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:52.458603 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:52.470069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730e2ab-42bd-43b1-a800-c883f9dfc704 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:52.470069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730e2ab-42bd-43b1-a800-c883f9dfc704 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:52.470069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730e2ab-42bd-43b1-a800-c883f9dfc704 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:52.470069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 353/691] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:37:52.477995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62717d-c302-4f3f-b9da-0a8618ce6cd2 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:52.478409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62717d-c302-4f3f-b9da-0a8618ce6cd2 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:52.490949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62717d-c302-4f3f-b9da-0a8618ce6cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:52.501761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:52.517257 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-8c62717d-c302-4f3f-b9da-0a8618ce6cd2 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:52.519754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8c62717d-c302-4f3f-b9da-0a8618ce6cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:52.584428 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89413f9a-2075-4db1-847c-3ff8e40f6eb7 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:52.608921 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 354/692] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:37:52.610064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62717d-c302-4f3f-b9da-0a8618ce6cd2 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:52.649879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a3463-d073-4209-b314-bb0e7d85da38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:52.650509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a3463-d073-4209-b314-bb0e7d85da38 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:52.650800 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a3463-d073-4209-b314-bb0e7d85da38 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:52.651070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a3463-d073-4209-b314-bb0e7d85da38 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:52.651812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a3463-d073-4209-b314-bb0e7d85da38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:52.652183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a3463-d073-4209-b314-bb0e7d85da38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:52.653624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a3463-d073-4209-b314-bb0e7d85da38 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:52.675283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 339/693] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:52.681547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a3463-d073-4209-b314-bb0e7d85da38 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:52.794268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62717d-c302-4f3f-b9da-0a8618ce6cd2 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:52.825418 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:52.828573 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dae97-aca7-4120-a2e6-c19edb3c9bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:52.829308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dae97-aca7-4120-a2e6-c19edb3c9bbe None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:52.829686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dae97-aca7-4120-a2e6-c19edb3c9bbe None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:52.830056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dae97-aca7-4120-a2e6-c19edb3c9bbe None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:52.830776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dae97-aca7-4120-a2e6-c19edb3c9bbe None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:52.832390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dae97-aca7-4120-a2e6-c19edb3c9bbe None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-171dae97-aca7-4120-a2e6-c19edb3c9bbe None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:52.844036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:52.845092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:37:52.845963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:52.847197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:52.850080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 355/694] 10.222.0.46 () {56 vars in 1129 bytes} [Fri Sep 9 12:37:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:52.851576 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dae97-aca7-4120-a2e6-c19edb3c9bbe None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:52.862407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a3463-d073-4209-b314-bb0e7d85da38 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:52.950460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d0ba2-4e8c-4119-ad3c-76388cb61649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:52.951430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d0ba2-4e8c-4119-ad3c-76388cb61649 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:52.951907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d0ba2-4e8c-4119-ad3c-76388cb61649 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:52.952436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d0ba2-4e8c-4119-ad3c-76388cb61649 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:52.953348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d0ba2-4e8c-4119-ad3c-76388cb61649 None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=e8e4805f0b1e484c98fb399313f88ef2) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:52.953862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d0ba2-4e8c-4119-ad3c-76388cb61649 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:52.955511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d0ba2-4e8c-4119-ad3c-76388cb61649 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:52.962596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 340/695] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:52 2022] GET /identity/v3/role_assignments?user.id=e8e4805f0b1e484c98fb399313f88ef2&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:52.963694 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d0ba2-4e8c-4119-ad3c-76388cb61649 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:52.966892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dae97-aca7-4120-a2e6-c19edb3c9bbe None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:53.023869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c1bb1-7d89-4df1-ae11-be04e4c69a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:53.024498 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c1bb1-7d89-4df1-ae11-be04e4c69a76 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:53.024812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c1bb1-7d89-4df1-ae11-be04e4c69a76 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:53.025137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c1bb1-7d89-4df1-ae11-be04e4c69a76 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:53.027881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c1bb1-7d89-4df1-ae11-be04e4c69a76 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:53.028269 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c1bb1-7d89-4df1-ae11-be04e4c69a76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:53.029698 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c1bb1-7d89-4df1-ae11-be04e4c69a76 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:53.040691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 356/696] 10.222.0.46 () {56 vars in 1124 bytes} [Fri Sep 9 12:37:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:53.047245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c1bb1-7d89-4df1-ae11-be04e4c69a76 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:53.158882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:53.164007 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d0ba2-4e8c-4119-ad3c-76388cb61649 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:53.212232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21f269-53e5-4537-8a1c-26bde1e8130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:53.212801 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21f269-53e5-4537-8a1c-26bde1e8130c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:53.213082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21f269-53e5-4537-8a1c-26bde1e8130c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:53.213355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21f269-53e5-4537-8a1c-26bde1e8130c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:53.223082 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c1bb1-7d89-4df1-ae11-be04e4c69a76 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:53.227686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:53.234180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21f269-53e5-4537-8a1c-26bde1e8130c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-5b21f269-53e5-4537-8a1c-26bde1e8130c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:53.240687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:53.242670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:53.243544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:53.243544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:53.243544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:53.243544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:53.243544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:53.243544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:53.243544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:53.243544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:53.243544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:53.243544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:53.243544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:37:53.243544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:53.244207 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 341/697] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:37:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:53.250345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21f269-53e5-4537-8a1c-26bde1e8130c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:53.267469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd0d3d-d2e5-4559-a130-f6c0d0ea918a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:53.268769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd0d3d-d2e5-4559-a130-f6c0d0ea918a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:53.269530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd0d3d-d2e5-4559-a130-f6c0d0ea918a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:53.270002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd0d3d-d2e5-4559-a130-f6c0d0ea918a None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:53.270913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd0d3d-d2e5-4559-a130-f6c0d0ea918a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:53.271410 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd0d3d-d2e5-4559-a130-f6c0d0ea918a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:53.274144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd0d3d-d2e5-4559-a130-f6c0d0ea918a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:53.281093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 357/698] 10.222.0.46 () {56 vars in 1124 bytes} [Fri Sep 9 12:37:53 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:53.374225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21f269-53e5-4537-8a1c-26bde1e8130c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:53.412471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113c732-c8bf-4b1b-8488-b71ef3977d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:53.413093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113c732-c8bf-4b1b-8488-b71ef3977d67 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:53.414252 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113c732-c8bf-4b1b-8488-b71ef3977d67 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:53.414252 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113c732-c8bf-4b1b-8488-b71ef3977d67 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:53.414808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113c732-c8bf-4b1b-8488-b71ef3977d67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:53.415114 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113c732-c8bf-4b1b-8488-b71ef3977d67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:53.416518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113c732-c8bf-4b1b-8488-b71ef3977d67 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:53.426556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 342/699] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:53.432037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd0d3d-d2e5-4559-a130-f6c0d0ea918a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:53.552801 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd0d3d-d2e5-4559-a130-f6c0d0ea918a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:53.581684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:53.599216 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2e671-bb4d-4be3-89d9-81035c56baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:53.599868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2e671-bb4d-4be3-89d9-81035c56baf5 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:53.600169 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2e671-bb4d-4be3-89d9-81035c56baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:53.600430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2e671-bb4d-4be3-89d9-81035c56baf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:53.615549 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2e671-bb4d-4be3-89d9-81035c56baf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:53.617615 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2e671-bb4d-4be3-89d9-81035c56baf5 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-42e2e671-bb4d-4be3-89d9-81035c56baf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:53.639295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:53.640305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:53.641238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:53.641238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:53.641238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:53.641238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 358/700] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:53.646327 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113c732-c8bf-4b1b-8488-b71ef3977d67 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:53.757177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:53.816744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113c732-c8bf-4b1b-8488-b71ef3977d67 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:53.855881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c1719-972a-4ec1-84ad-7786e46b26f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:53.856494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c1719-972a-4ec1-84ad-7786e46b26f3 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:53.856776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c1719-972a-4ec1-84ad-7786e46b26f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:53.857053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c1719-972a-4ec1-84ad-7786e46b26f3 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:53.857694 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c1719-972a-4ec1-84ad-7786e46b26f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:53.857996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c1719-972a-4ec1-84ad-7786e46b26f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:53.859447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c1719-972a-4ec1-84ad-7786e46b26f3 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:53.872262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 343/701] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:53.877714 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2e671-bb4d-4be3-89d9-81035c56baf5 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:53.992764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2e671-bb4d-4be3-89d9-81035c56baf5 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:54.043915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26116d-a307-460b-a128-90302d078b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:54.044611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26116d-a307-460b-a128-90302d078b2b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:54.044869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26116d-a307-460b-a128-90302d078b2b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:54.045148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26116d-a307-460b-a128-90302d078b2b None admin] PATH_INFO: `/v3/users/nova` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:54.055851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26116d-a307-460b-a128-90302d078b2b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:54.057283 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26116d-a307-460b-a128-90302d078b2b None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-4f26116d-a307-460b-a128-90302d078b2b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:54.073786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:54.074881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:54.075834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:54.075834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:54.075834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:54.075834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:54.075834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:54.075834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:54.075834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:54.075834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:54.075834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:37:54.075834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:54.075834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 359/702] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 12:37:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:54.078067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c1719-972a-4ec1-84ad-7786e46b26f3 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:54.225520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c1719-972a-4ec1-84ad-7786e46b26f3 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:54.258515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:54.273260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c2ab6-18cc-40d0-a9be-c6aeaacfe041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:54.277782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c2ab6-18cc-40d0-a9be-c6aeaacfe041 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:54.277782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c2ab6-18cc-40d0-a9be-c6aeaacfe041 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:54.277782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c2ab6-18cc-40d0-a9be-c6aeaacfe041 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:54.277782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c2ab6-18cc-40d0-a9be-c6aeaacfe041 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:54.277782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c2ab6-18cc-40d0-a9be-c6aeaacfe041 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:54.277782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c2ab6-18cc-40d0-a9be-c6aeaacfe041 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:54.302808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26116d-a307-460b-a128-90302d078b2b None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:54.304710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 344/703] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 12:37:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:54.306910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f7532-17ec-4986-8623-e056b8ad7d5e None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:54.307186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f7532-17ec-4986-8623-e056b8ad7d5e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:54.307471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f7532-17ec-4986-8623-e056b8ad7d5e None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:54.309048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 345/704] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:54.322763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f2051-1ff3-49ff-a669-ea69ea997888 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:54.323142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f2051-1ff3-49ff-a669-ea69ea997888 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:54.323316 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f2051-1ff3-49ff-a669-ea69ea997888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:54.342302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-f90f2051-1ff3-49ff-a669-ea69ea997888 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:54.343564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f90f2051-1ff3-49ff-a669-ea69ea997888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:54.394336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:54.431463 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 346/705] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:37:54.433498 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f9ffa-75ce-4765-8f73-43bfb38f0cb8 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:54.433929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f9ffa-75ce-4765-8f73-43bfb38f0cb8 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:54.434206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f9ffa-75ce-4765-8f73-43bfb38f0cb8 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:54.436648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 347/706] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:54.448127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc424e15-0843-4842-a566-8b5c9107b53c None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:54.448127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc424e15-0843-4842-a566-8b5c9107b53c None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:54.448488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc424e15-0843-4842-a566-8b5c9107b53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:54.479634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-dc424e15-0843-4842-a566-8b5c9107b53c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:54.480849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-dc424e15-0843-4842-a566-8b5c9107b53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:54.482643 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26116d-a307-460b-a128-90302d078b2b None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:54.537348 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73afde-9e8c-4d45-bb67-08ad05eea819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:54.537948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73afde-9e8c-4d45-bb67-08ad05eea819 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:54.538899 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73afde-9e8c-4d45-bb67-08ad05eea819 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:54.539068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73afde-9e8c-4d45-bb67-08ad05eea819 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:54.598793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 348/707] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:54.655980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73afde-9e8c-4d45-bb67-08ad05eea819 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:54.657401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73afde-9e8c-4d45-bb67-08ad05eea819 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:54.716845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc424e15-0843-4842-a566-8b5c9107b53c None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:54.728124 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:54.782311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 360/708] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:54.787043 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73afde-9e8c-4d45-bb67-08ad05eea819 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:54.809770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:54.895203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc424e15-0843-4842-a566-8b5c9107b53c None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:54.915877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73afde-9e8c-4d45-bb67-08ad05eea819 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:54.932784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ca960-1132-405f-8fa6-b2383bf6733e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:54.933862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ca960-1132-405f-8fa6-b2383bf6733e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:54.934370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ca960-1132-405f-8fa6-b2383bf6733e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:54.934935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ca960-1132-405f-8fa6-b2383bf6733e None admin] PATH_INFO: `/v3/services/network` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:54.935986 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ca960-1132-405f-8fa6-b2383bf6733e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:54.937619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ca960-1132-405f-8fa6-b2383bf6733e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-5b6ca960-1132-405f-8fa6-b2383bf6733e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:54.950085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:54.951148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:37:54.952001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:54.952758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d685a0-a88f-404d-919e-791218f94306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:54.952758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d685a0-a88f-404d-919e-791218f94306 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:54.952758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d685a0-a88f-404d-919e-791218f94306 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:54.952758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d685a0-a88f-404d-919e-791218f94306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:54.955286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 349/709] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:54 2022] GET /identity/v3/services/network => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:54.956302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ca960-1132-405f-8fa6-b2383bf6733e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:54.961729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:54.964012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d685a0-a88f-404d-919e-791218f94306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:54.965395 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d685a0-a88f-404d-919e-791218f94306 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-82d685a0-a88f-404d-919e-791218f94306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:54.980330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:54.982420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:54.983413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:54.983413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:54.983413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:54.983413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 361/710] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:37:54.983413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d685a0-a88f-404d-919e-791218f94306 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:55.071569 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ca960-1132-405f-8fa6-b2383bf6733e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:55.074825 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:55.085539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d685a0-a88f-404d-919e-791218f94306 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:55.113429 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4406e-cf53-4e44-a042-c08d98367764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:55.114385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4406e-cf53-4e44-a042-c08d98367764 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:55.114915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4406e-cf53-4e44-a042-c08d98367764 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:55.115462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4406e-cf53-4e44-a042-c08d98367764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:55.128242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4406e-cf53-4e44-a042-c08d98367764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:55.129111 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f619e-f0d6-48b7-a64f-d2d85d385895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:55.129715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f619e-f0d6-48b7-a64f-d2d85d385895 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:55.130051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f619e-f0d6-48b7-a64f-d2d85d385895 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:55.130320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f619e-f0d6-48b7-a64f-d2d85d385895 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:55.130848 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4406e-cf53-4e44-a042-c08d98367764 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:55.130955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f619e-f0d6-48b7-a64f-d2d85d385895 None admin] RBAC: Adding query filter params (name=network) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:55.131304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f619e-f0d6-48b7-a64f-d2d85d385895 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:55.133280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f619e-f0d6-48b7-a64f-d2d85d385895 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:55.138018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 362/711] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 12:37:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:55.138884 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f619e-f0d6-48b7-a64f-d2d85d385895 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-61c4406e-cf53-4e44-a042-c08d98367764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:55.148051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:55.149017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:55.150018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:55.150018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:55.150018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:55.152274 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 350/712] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:55.153348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4406e-cf53-4e44-a042-c08d98367764 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:55.194389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:55.238001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f619e-f0d6-48b7-a64f-d2d85d385895 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:55.262711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4406e-cf53-4e44-a042-c08d98367764 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:55.273182 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f18c6-738a-4083-9f66-fa116dcaa797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:55.273636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f18c6-738a-4083-9f66-fa116dcaa797 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:55.273912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f18c6-738a-4083-9f66-fa116dcaa797 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:55.274181 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f18c6-738a-4083-9f66-fa116dcaa797 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:55.274852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f18c6-738a-4083-9f66-fa116dcaa797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:55.275151 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f18c6-738a-4083-9f66-fa116dcaa797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:55.276495 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f18c6-738a-4083-9f66-fa116dcaa797 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:55.293054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 363/713] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:55.294049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f18c6-738a-4083-9f66-fa116dcaa797 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:55.301567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591f24d-fadf-4ffe-997c-d974f4814cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:55.302546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591f24d-fadf-4ffe-997c-d974f4814cdc None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:55.303105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591f24d-fadf-4ffe-997c-d974f4814cdc None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:55.303604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591f24d-fadf-4ffe-997c-d974f4814cdc None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:55.305195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591f24d-fadf-4ffe-997c-d974f4814cdc None admin] RBAC: Adding query filter params (type=network) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:55.305799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591f24d-fadf-4ffe-997c-d974f4814cdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:55.307525 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591f24d-fadf-4ffe-997c-d974f4814cdc None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:55.314818 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 351/714] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 12:37:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:55.315851 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591f24d-fadf-4ffe-997c-d974f4814cdc None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:55.342872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:55.406685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f18c6-738a-4083-9f66-fa116dcaa797 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:55.424876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:55.433554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591f24d-fadf-4ffe-997c-d974f4814cdc None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:55.445720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e4c2d-260b-42de-8645-164eae421246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:55.446598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e4c2d-260b-42de-8645-164eae421246 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:55.446713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e4c2d-260b-42de-8645-164eae421246 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:55.446906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e4c2d-260b-42de-8645-164eae421246 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:55.447540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e4c2d-260b-42de-8645-164eae421246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:55.447836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e4c2d-260b-42de-8645-164eae421246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:55.449185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e4c2d-260b-42de-8645-164eae421246 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:55.461793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 364/715] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:55.467475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e4c2d-260b-42de-8645-164eae421246 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:55.474991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c963cd5-2472-4141-b09f-33590cf183f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:55.476391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c963cd5-2472-4141-b09f-33590cf183f6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:55.476829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c963cd5-2472-4141-b09f-33590cf183f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:55.477251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c963cd5-2472-4141-b09f-33590cf183f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:55.485430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c963cd5-2472-4141-b09f-33590cf183f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:55.487257 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c963cd5-2472-4141-b09f-33590cf183f6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-6c963cd5-2472-4141-b09f-33590cf183f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:55.500858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:55.501862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:55.502794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:55.502794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:55.502794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:55.502794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:55.502794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:55.502794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:55.502794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:55.502794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:55.505465 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 352/716] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:55.513078 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c963cd5-2472-4141-b09f-33590cf183f6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:55.584493 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e4c2d-260b-42de-8645-164eae421246 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:55.643771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b736a-7a93-43cc-9714-d6fa972d9461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:55.644387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b736a-7a93-43cc-9714-d6fa972d9461 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:55.644665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b736a-7a93-43cc-9714-d6fa972d9461 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:55.644935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b736a-7a93-43cc-9714-d6fa972d9461 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:55.645591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b736a-7a93-43cc-9714-d6fa972d9461 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:55.647046 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b736a-7a93-43cc-9714-d6fa972d9461 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:55.673383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:55.694039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c963cd5-2472-4141-b09f-33590cf183f6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:55.735655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8271e-3e36-4624-8eec-b8fd5ff2de0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:55.737127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8271e-3e36-4624-8eec-b8fd5ff2de0c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:55.737626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8271e-3e36-4624-8eec-b8fd5ff2de0c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:55.738139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8271e-3e36-4624-8eec-b8fd5ff2de0c None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:55.739044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8271e-3e36-4624-8eec-b8fd5ff2de0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:55.739588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8271e-3e36-4624-8eec-b8fd5ff2de0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:55.741221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8271e-3e36-4624-8eec-b8fd5ff2de0c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:55.749599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 365/717] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:37:55 2022] POST /identity/v3/users => generated 254 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:37:55.764654 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 353/718] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:55.770886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b736a-7a93-43cc-9714-d6fa972d9461 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:55.817795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:55.921234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b736a-7a93-43cc-9714-d6fa972d9461 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:55.966209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfbeb1-d651-4f7d-91a4-c6b98fbdfd57 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:55.966539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfbeb1-d651-4f7d-91a4-c6b98fbdfd57 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:55.966796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfbeb1-d651-4f7d-91a4-c6b98fbdfd57 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:55.968393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 354/719] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:37:55.970334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1a5de-2ddb-49f0-aac5-8dc2c5c664f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:55.970998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1a5de-2ddb-49f0-aac5-8dc2c5c664f4 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:55.971270 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1a5de-2ddb-49f0-aac5-8dc2c5c664f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:55.971538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1a5de-2ddb-49f0-aac5-8dc2c5c664f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:55.972366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1a5de-2ddb-49f0-aac5-8dc2c5c664f4 None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=f29af98c5fba4569b16edf240b13c5c5) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:55.972664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1a5de-2ddb-49f0-aac5-8dc2c5c664f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:55.973838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321a65e-83dc-4abb-b905-78c3fcbaac30 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:55.974101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1a5de-2ddb-49f0-aac5-8dc2c5c664f4 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:55.974206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321a65e-83dc-4abb-b905-78c3fcbaac30 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:55.974392 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321a65e-83dc-4abb-b905-78c3fcbaac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:55.984610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 366/720] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:55 2022] GET /identity/v3/role_assignments?user.id=f29af98c5fba4569b16edf240b13c5c5&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:37:55.994548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-1321a65e-83dc-4abb-b905-78c3fcbaac30 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:55.995766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-1321a65e-83dc-4abb-b905-78c3fcbaac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:56.080591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 355/721] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:56.176207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f701e-0d2c-4dc3-9e52-82463cb056d4 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:56.176726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f701e-0d2c-4dc3-9e52-82463cb056d4 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:56.177042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f701e-0d2c-4dc3-9e52-82463cb056d4 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:56.178828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 367/722] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:56.184545 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913af1c7-622f-4a59-9290-424c9c492a6d None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:56.184838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913af1c7-622f-4a59-9290-424c9c492a6d None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:56.185373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913af1c7-622f-4a59-9290-424c9c492a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:56.205668 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-913af1c7-622f-4a59-9290-424c9c492a6d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:56.207242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-913af1c7-622f-4a59-9290-424c9c492a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:56.289193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:56.300052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 356/723] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:56.365102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f701e-0d2c-4dc3-9e52-82463cb056d4 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:56.500882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:56.509800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f701e-0d2c-4dc3-9e52-82463cb056d4 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:56.558459 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa2ce8-ca30-432f-8c96-fb7bc15d7e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:56.559054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2ce8-ca30-432f-8c96-fb7bc15d7e3c None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:56.559331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2ce8-ca30-432f-8c96-fb7bc15d7e3c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:56.559641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2ce8-ca30-432f-8c96-fb7bc15d7e3c None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:56.560566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa2ce8-ca30-432f-8c96-fb7bc15d7e3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:56.561916 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa2ce8-ca30-432f-8c96-fb7bc15d7e3c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:56.679600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913af1c7-622f-4a59-9290-424c9c492a6d None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:56.694633 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 368/724] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:37:56 2022] POST /identity/v3/services => generated 251 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:37:56.830256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913af1c7-622f-4a59-9290-424c9c492a6d None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:56.888494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c238a95-94a9-43d2-a5be-8e1ec7133b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:56.889128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c238a95-94a9-43d2-a5be-8e1ec7133b82 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:56.889402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c238a95-94a9-43d2-a5be-8e1ec7133b82 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:56.889665 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c238a95-94a9-43d2-a5be-8e1ec7133b82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:56.905831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c238a95-94a9-43d2-a5be-8e1ec7133b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-8c238a95-94a9-43d2-a5be-8e1ec7133b82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:56.910619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:56.911548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:56.912957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:56.912957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:56.912957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:56.912957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:56.912957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:56.912957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:56.912957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:56.912957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:56.912957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:56.912957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:56.912957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:37:56.912957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:56.912957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 357/725] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:37:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:37:56.918517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa2ce8-ca30-432f-8c96-fb7bc15d7e3c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:57.028868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa2ce8-ca30-432f-8c96-fb7bc15d7e3c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:57.073868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aae60f-1aac-4461-9399-eadb7130b107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:57.074869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aae60f-1aac-4461-9399-eadb7130b107 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:57.075359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aae60f-1aac-4461-9399-eadb7130b107 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:57.075915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aae60f-1aac-4461-9399-eadb7130b107 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:57.076784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aae60f-1aac-4461-9399-eadb7130b107 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:57.077313 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aae60f-1aac-4461-9399-eadb7130b107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:57.078884 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aae60f-1aac-4461-9399-eadb7130b107 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:57.087837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 369/726] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:57.093675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c238a95-94a9-43d2-a5be-8e1ec7133b82 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:57.187703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:57.239389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c238a95-94a9-43d2-a5be-8e1ec7133b82 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:57.273139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504a88e-19fd-4367-948d-0f7ca4bdaee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:57.273936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504a88e-19fd-4367-948d-0f7ca4bdaee3 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:57.274215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504a88e-19fd-4367-948d-0f7ca4bdaee3 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:57.274496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504a88e-19fd-4367-948d-0f7ca4bdaee3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:57.284447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504a88e-19fd-4367-948d-0f7ca4bdaee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:57.285879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504a88e-19fd-4367-948d-0f7ca4bdaee3 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-b504a88e-19fd-4367-948d-0f7ca4bdaee3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:57.300125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:57.301118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:57.302361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:57.302361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:57.302361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:57.302361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 358/727] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:57.307477 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aae60f-1aac-4461-9399-eadb7130b107 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:57.352883 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:57.455500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aae60f-1aac-4461-9399-eadb7130b107 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:57.503822 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4ad22-1c8e-4e41-a4b1-315fea3bfdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:57.504812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4ad22-1c8e-4e41-a4b1-315fea3bfdf2 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:57.505318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4ad22-1c8e-4e41-a4b1-315fea3bfdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:57.505803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4ad22-1c8e-4e41-a4b1-315fea3bfdf2 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:57.506681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4ad22-1c8e-4e41-a4b1-315fea3bfdf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:57.507273 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4ad22-1c8e-4e41-a4b1-315fea3bfdf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:57.508872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4ad22-1c8e-4e41-a4b1-315fea3bfdf2 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:57.525918 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 370/728] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:57.531425 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504a88e-19fd-4367-948d-0f7ca4bdaee3 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:57.673488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504a88e-19fd-4367-948d-0f7ca4bdaee3 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:57.688870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:57.723761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a0d0a-86f4-4ab3-86a1-a51feea290f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:57.724395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a0d0a-86f4-4ab3-86a1-a51feea290f1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:57.724671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a0d0a-86f4-4ab3-86a1-a51feea290f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:57.724940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a0d0a-86f4-4ab3-86a1-a51feea290f1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:57.736119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a0d0a-86f4-4ab3-86a1-a51feea290f1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:57.737621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a0d0a-86f4-4ab3-86a1-a51feea290f1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-8e1a0d0a-86f4-4ab3-86a1-a51feea290f1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:37:57.752329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:57.753436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:57.754838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:57.754838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:57.754838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:37:57.754838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:37:57.754838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:37:57.754838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:37:57.754838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:37:57.754838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:37:57.754838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:37:57.754838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:57.754838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 359/729] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:37:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:57.765275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4ad22-1c8e-4e41-a4b1-315fea3bfdf2 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:57.884901 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:57.906556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4ad22-1c8e-4e41-a4b1-315fea3bfdf2 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:57.948614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4f552-6dc0-48aa-87e6-2a7eb40a821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:57.949579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4f552-6dc0-48aa-87e6-2a7eb40a821a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:57.950061 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4f552-6dc0-48aa-87e6-2a7eb40a821a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:57.950551 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4f552-6dc0-48aa-87e6-2a7eb40a821a None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:57.951531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4f552-6dc0-48aa-87e6-2a7eb40a821a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:57.952077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4f552-6dc0-48aa-87e6-2a7eb40a821a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:57.953673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4f552-6dc0-48aa-87e6-2a7eb40a821a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:57.971423 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c10980-5742-4361-9717-58da0d33b338 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:57.971923 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c10980-5742-4361-9717-58da0d33b338 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:57.972028 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c10980-5742-4361-9717-58da0d33b338 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:57.973427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 360/730] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:57.977960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 371/731] 10.222.0.46 () {56 vars in 1135 bytes} [Fri Sep 9 12:37:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:57.981948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf5e92-4edd-4df5-a593-100efd1d0158 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:57.981948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf5e92-4edd-4df5-a593-100efd1d0158 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:57.981948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf5e92-4edd-4df5-a593-100efd1d0158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:57.983379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4f552-6dc0-48aa-87e6-2a7eb40a821a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:58.015585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-3baf5e92-4edd-4df5-a593-100efd1d0158 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:58.016998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3baf5e92-4edd-4df5-a593-100efd1d0158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:58.132684 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 361/732] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:37:58.147084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4f552-6dc0-48aa-87e6-2a7eb40a821a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:58.185904 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56283f16-3ee6-42fe-bf7b-c058352f6856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:58.186956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56283f16-3ee6-42fe-bf7b-c058352f6856 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:58.187438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56283f16-3ee6-42fe-bf7b-c058352f6856 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:58.187980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56283f16-3ee6-42fe-bf7b-c058352f6856 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:58.192193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:58.266531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56283f16-3ee6-42fe-bf7b-c058352f6856 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:58.266531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56283f16-3ee6-42fe-bf7b-c058352f6856 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:58.371324 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 372/733] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:37:57 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:37:58.379587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf5e92-4edd-4df5-a593-100efd1d0158 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:58.539646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:58.544484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf5e92-4edd-4df5-a593-100efd1d0158 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:58.632172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2956ef6-26c1-429f-830b-61d4d70a4205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:58.632960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2956ef6-26c1-429f-830b-61d4d70a4205 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:58.633239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2956ef6-26c1-429f-830b-61d4d70a4205 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:58.633510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2956ef6-26c1-429f-830b-61d4d70a4205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:58.655167 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2956ef6-26c1-429f-830b-61d4d70a4205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:58.662801 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2956ef6-26c1-429f-830b-61d4d70a4205 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:58.664150 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56283f16-3ee6-42fe-bf7b-c058352f6856 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-c2956ef6-26c1-429f-830b-61d4d70a4205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:37:58.689365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:58.690454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:37:58.703485 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:37:58.703485 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:37:58.703485 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:58.703485 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 362/734] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:58.703485 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2956ef6-26c1-429f-830b-61d4d70a4205 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:58.709157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:58.790330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56283f16-3ee6-42fe-bf7b-c058352f6856 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:58.829862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e81244-9988-41e0-b89d-df65156180a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:58.830850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e81244-9988-41e0-b89d-df65156180a9 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:58.831438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e81244-9988-41e0-b89d-df65156180a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:58.831914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e81244-9988-41e0-b89d-df65156180a9 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:58.832941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e81244-9988-41e0-b89d-df65156180a9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:58.834628 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e81244-9988-41e0-b89d-df65156180a9 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:58.892812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2956ef6-26c1-429f-830b-61d4d70a4205 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:58.920940 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 373/735] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:37:58 2022] POST /identity/v3/services => generated 240 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:37:58.923377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8166c5-aac2-4258-ab5a-5ff6445ece78 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:58.923665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8166c5-aac2-4258-ab5a-5ff6445ece78 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:58.923945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8166c5-aac2-4258-ab5a-5ff6445ece78 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:58.926853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 374/736] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:58.932976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b585b-467c-4be8-8182-da0bc8e1e8a1 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:58.933272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b585b-467c-4be8-8182-da0bc8e1e8a1 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:58.933531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b585b-467c-4be8-8182-da0bc8e1e8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:58.950247 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b026dbe-62ee-4a68-8c4c-85529aa29b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:58.951371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b026dbe-62ee-4a68-8c4c-85529aa29b1e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:58.951880 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b026dbe-62ee-4a68-8c4c-85529aa29b1e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:58.952359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b026dbe-62ee-4a68-8c4c-85529aa29b1e None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:58.953340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b026dbe-62ee-4a68-8c4c-85529aa29b1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:58.953967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b026dbe-62ee-4a68-8c4c-85529aa29b1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:58.955631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b026dbe-62ee-4a68-8c4c-85529aa29b1e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:58.970614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 363/737] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:58.976457 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b026dbe-62ee-4a68-8c4c-85529aa29b1e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:58.979232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-ae5b585b-467c-4be8-8182-da0bc8e1e8a1 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:58.986917 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ae5b585b-467c-4be8-8182-da0bc8e1e8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:58.995756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:59.090644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 375/738] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:37:59.091879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9a784-3a74-4ed6-af16-85a77b4fc0be None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:59.092159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9a784-3a74-4ed6-af16-85a77b4fc0be None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:59.092429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9a784-3a74-4ed6-af16-85a77b4fc0be None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:59.097984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 376/739] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:37:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:37:59.099322 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ed4e2-deff-4ff2-9d35-4d51c3562139 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:59.105157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ed4e2-deff-4ff2-9d35-4d51c3562139 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:59.105516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ed4e2-deff-4ff2-9d35-4d51c3562139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:59.133124 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-e63ed4e2-deff-4ff2-9d35-4d51c3562139 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:37:59.134306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e63ed4e2-deff-4ff2-9d35-4d51c3562139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:37:59.141836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b026dbe-62ee-4a68-8c4c-85529aa29b1e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:59.158918 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:59.181896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2e6b5-1646-4470-8d43-476e493b0cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:59.182562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2e6b5-1646-4470-8d43-476e493b0cfd None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:59.182903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2e6b5-1646-4470-8d43-476e493b0cfd None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:59.183177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2e6b5-1646-4470-8d43-476e493b0cfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:59.190468 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2e6b5-1646-4470-8d43-476e493b0cfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:59.191980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2e6b5-1646-4470-8d43-476e493b0cfd None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:59.203636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 377/740] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:37:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-26c2e6b5-1646-4470-8d43-476e493b0cfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:37:59.204678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:59.209783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:59.210948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:37:59.210948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:37:59.210948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:37:59.210948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:37:59.210948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:37:59.210948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:37:59.210948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:37:59.210948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:59.210948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 364/741] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:37:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:59.211444 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ed4e2-deff-4ff2-9d35-4d51c3562139 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:59.315166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ed4e2-deff-4ff2-9d35-4d51c3562139 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:59.348985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b717c8-2c25-46f3-a456-68ff5852c814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:59.349503 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b717c8-2c25-46f3-a456-68ff5852c814 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:59.349782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b717c8-2c25-46f3-a456-68ff5852c814 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:59.350049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b717c8-2c25-46f3-a456-68ff5852c814 None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:59.351338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b717c8-2c25-46f3-a456-68ff5852c814 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:59.351642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b717c8-2c25-46f3-a456-68ff5852c814 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:59.353145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b717c8-2c25-46f3-a456-68ff5852c814 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:59.365050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 378/742] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:37:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:59.370711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2e6b5-1646-4470-8d43-476e493b0cfd None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:59.401505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b717c8-2c25-46f3-a456-68ff5852c814 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:59.420964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:59.497963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:59.548270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2e6b5-1646-4470-8d43-476e493b0cfd None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:59.553345 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b717c8-2c25-46f3-a456-68ff5852c814 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:59.593687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aaedc5-b4a2-495d-af25-d90772409372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:59.594297 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaedc5-b4a2-495d-af25-d90772409372 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:59.594607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaedc5-b4a2-495d-af25-d90772409372 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:59.594970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaedc5-b4a2-495d-af25-d90772409372 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:59.595804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aaedc5-b4a2-495d-af25-d90772409372 None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=e8e4805f0b1e484c98fb399313f88ef2) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:59.596114 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aaedc5-b4a2-495d-af25-d90772409372 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:59.596387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a3d19-8115-42b0-9663-914e748a811d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:59.597314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a3d19-8115-42b0-9663-914e748a811d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:59.597600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a3d19-8115-42b0-9663-914e748a811d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:59.597873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a3d19-8115-42b0-9663-914e748a811d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:59.598701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aaedc5-b4a2-495d-af25-d90772409372 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:59.606194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 365/743] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:37:59 2022] GET /identity/v3/role_assignments?user.id=e8e4805f0b1e484c98fb399313f88ef2&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:59.607876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aaedc5-b4a2-495d-af25-d90772409372 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:59.610427 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a3d19-8115-42b0-9663-914e748a811d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-651a3d19-8115-42b0-9663-914e748a811d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:59.618263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:59.619318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:59.620162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:59.620162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:59.620162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:59.620162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:59.620162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:59.620162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:59.620162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:59.620162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:59.620162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:59.620162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:59.620162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:37:59.620162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:37:59.620162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 379/744] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:37:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:37:59.625942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a3d19-8115-42b0-9663-914e748a811d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:59.726829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:59.744512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aaedc5-b4a2-495d-af25-d90772409372 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:59.761309 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a3d19-8115-42b0-9663-914e748a811d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:59.779177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:37:59.786427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ea1cb-0db4-42c3-be4b-87197debb4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:59.787203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ea1cb-0db4-42c3-be4b-87197debb4c0 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:59.787507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ea1cb-0db4-42c3-be4b-87197debb4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:59.787828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ea1cb-0db4-42c3-be4b-87197debb4c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:59.800987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ea1cb-0db4-42c3-be4b-87197debb4c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:37:59.806792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24bcc7-a84b-4e4d-aace-a02fef977ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:37:59.807765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24bcc7-a84b-4e4d-aace-a02fef977ed0 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:37:59.808226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24bcc7-a84b-4e4d-aace-a02fef977ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-d30ea1cb-0db4-42c3-be4b-87197debb4c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:37:59.808331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:37:59.809196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:37:59.810062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:37:59.810062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:37:59.810062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:37:59.810062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:37:59.810062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:37:59.810062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:37:59.810062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:37:59.810062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:37:59.810062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:37:59.810062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:37:59.810062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:37:59.810062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:37:59.810062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 366/745] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:37:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:37:59.811852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24bcc7-a84b-4e4d-aace-a02fef977ed0 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:37:59.813291 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24bcc7-a84b-4e4d-aace-a02fef977ed0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:37:59.813983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24bcc7-a84b-4e4d-aace-a02fef977ed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:37:59.815930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24bcc7-a84b-4e4d-aace-a02fef977ed0 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:37:59.816456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ea1cb-0db4-42c3-be4b-87197debb4c0 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:59.829356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 380/746] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:37:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:37:59.837531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24bcc7-a84b-4e4d-aace-a02fef977ed0 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:37:59.960148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24bcc7-a84b-4e4d-aace-a02fef977ed0 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:37:59.987038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ea1cb-0db4-42c3-be4b-87197debb4c0 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:00.004080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024132fb-52be-4f57-9e22-55484669062a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:00.005105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024132fb-52be-4f57-9e22-55484669062a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:00.005840 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024132fb-52be-4f57-9e22-55484669062a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:00.006255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024132fb-52be-4f57-9e22-55484669062a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:00.015862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:00.020636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024132fb-52be-4f57-9e22-55484669062a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:00.022256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024132fb-52be-4f57-9e22-55484669062a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:00.024741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0aa8fd-614f-4eb9-8c4b-09cb808855ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:00.025354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0aa8fd-614f-4eb9-8c4b-09cb808855ba None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:00.025633 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0aa8fd-614f-4eb9-8c4b-09cb808855ba None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:00.025905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0aa8fd-614f-4eb9-8c4b-09cb808855ba None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:00.026665 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0aa8fd-614f-4eb9-8c4b-09cb808855ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:00.026971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0aa8fd-614f-4eb9-8c4b-09cb808855ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:00.028340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0aa8fd-614f-4eb9-8c4b-09cb808855ba None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:00.037156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 367/747] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:37:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:00.039031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49885596-9715-490a-b314-d14cefb0aca2 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:00.039316 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49885596-9715-490a-b314-d14cefb0aca2 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:00.044025 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49885596-9715-490a-b314-d14cefb0aca2 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-024132fb-52be-4f57-9e22-55484669062a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:00.044131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:00.045012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:00.045913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:00.045913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:00.045913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:00.046081 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 368/749] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:00.046081 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49885596-9715-490a-b314-d14cefb0aca2 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:00.046674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 381/749] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:00.048341 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcd173-ebba-418a-abc3-3a160d9134f5 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:00.048818 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcd173-ebba-418a-abc3-3a160d9134f5 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:00.049544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcd173-ebba-418a-abc3-3a160d9134f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:00.074638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-a8bcd173-ebba-418a-abc3-3a160d9134f5 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:00.076537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a8bcd173-ebba-418a-abc3-3a160d9134f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:00.123033 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:00.176734 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49885596-9715-490a-b314-d14cefb0aca2 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:00.182004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 382/750] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:00.183243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcd173-ebba-418a-abc3-3a160d9134f5 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:00.255084 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfd779-000a-4a84-be98-28eef7eeed1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:00.255802 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfd779-000a-4a84-be98-28eef7eeed1e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:00.256085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfd779-000a-4a84-be98-28eef7eeed1e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:00.256363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfd779-000a-4a84-be98-28eef7eeed1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:00.279170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bfd779-000a-4a84-be98-28eef7eeed1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:00.280714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bfd779-000a-4a84-be98-28eef7eeed1e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:00.298125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcd173-ebba-418a-abc3-3a160d9134f5 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-d7bfd779-000a-4a84-be98-28eef7eeed1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:00.301308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:00.302239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:00.303149 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:00.303149 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:00.303149 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:00.305775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 369/751] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:00.313996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfd779-000a-4a84-be98-28eef7eeed1e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:00.333902 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:00.358571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc278b10-a34c-4071-84e0-10f9eae48da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:00.359237 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc278b10-a34c-4071-84e0-10f9eae48da2 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:00.359514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc278b10-a34c-4071-84e0-10f9eae48da2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:00.359785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc278b10-a34c-4071-84e0-10f9eae48da2 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:00.360493 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc278b10-a34c-4071-84e0-10f9eae48da2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:00.360793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc278b10-a34c-4071-84e0-10f9eae48da2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:00.362286 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc278b10-a34c-4071-84e0-10f9eae48da2 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:00.380427 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 383/752] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:00.386099 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc278b10-a34c-4071-84e0-10f9eae48da2 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:00.424272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfd779-000a-4a84-be98-28eef7eeed1e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:00.427075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:00.484512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a3c2d-a76b-45cb-8a79-124566b100d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:00.485533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3c2d-a76b-45cb-8a79-124566b100d8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:00.486181 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3c2d-a76b-45cb-8a79-124566b100d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:00.487665 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3c2d-a76b-45cb-8a79-124566b100d8 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:00.488507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a3c2d-a76b-45cb-8a79-124566b100d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:00.489182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a3c2d-a76b-45cb-8a79-124566b100d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:00.491471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a3c2d-a76b-45cb-8a79-124566b100d8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:00.513319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 370/753] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:00.523507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a3c2d-a76b-45cb-8a79-124566b100d8 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:00.549242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc278b10-a34c-4071-84e0-10f9eae48da2 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:00.590029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e51412-6b90-4f73-8030-ff4c23ede56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:00.590736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e51412-6b90-4f73-8030-ff4c23ede56a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:00.591009 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e51412-6b90-4f73-8030-ff4c23ede56a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:00.591266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e51412-6b90-4f73-8030-ff4c23ede56a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:00.601837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e51412-6b90-4f73-8030-ff4c23ede56a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:00.603382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e51412-6b90-4f73-8030-ff4c23ede56a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-f7e51412-6b90-4f73-8030-ff4c23ede56a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:00.616070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:00.617862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:00.618834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:00.618834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:00.618834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:00.618834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:00.618834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:00.618834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:00.618834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:00.618834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:00.618834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:38:00.618834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:00.618834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 384/754] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 12:38:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:00.619436 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e51412-6b90-4f73-8030-ff4c23ede56a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:00.639592 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a3c2d-a76b-45cb-8a79-124566b100d8 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:00.675150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1c4f0-25a3-471c-a1d1-1842bf9279dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:00.676142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1c4f0-25a3-471c-a1d1-1842bf9279dc None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:00.676617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1c4f0-25a3-471c-a1d1-1842bf9279dc None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:00.677101 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1c4f0-25a3-471c-a1d1-1842bf9279dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:00.685929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1c4f0-25a3-471c-a1d1-1842bf9279dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:00.686947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:00.687882 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1c4f0-25a3-471c-a1d1-1842bf9279dc None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-efd1c4f0-25a3-471c-a1d1-1842bf9279dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:00.699676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:00.704817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:00.705749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:00.705749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:00.705749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:00.705749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:00.705749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:00.705749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:00.705749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:00.705749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:00.705749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:38:00.705749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:00.705749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 371/755] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 12:38:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:00.705749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1c4f0-25a3-471c-a1d1-1842bf9279dc None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:00.731226 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:00.736765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e51412-6b90-4f73-8030-ff4c23ede56a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:00.773919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0807898-dde6-44fd-b43f-d7b0e4afe198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:00.774498 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0807898-dde6-44fd-b43f-d7b0e4afe198 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:00.774812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0807898-dde6-44fd-b43f-d7b0e4afe198 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:00.775084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0807898-dde6-44fd-b43f-d7b0e4afe198 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:00.775909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0807898-dde6-44fd-b43f-d7b0e4afe198 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:00.777416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0807898-dde6-44fd-b43f-d7b0e4afe198 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-f0807898-dde6-44fd-b43f-d7b0e4afe198 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:00.786433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:00.787437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:00.789045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 385/756] 10.222.0.46 () {56 vars in 1129 bytes} [Fri Sep 9 12:38:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:00.789831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0807898-dde6-44fd-b43f-d7b0e4afe198 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:00.812104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1c4f0-25a3-471c-a1d1-1842bf9279dc None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:00.848832 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf10c73-4cdd-4e32-970e-53638c971230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:00.849788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf10c73-4cdd-4e32-970e-53638c971230 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:00.850264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf10c73-4cdd-4e32-970e-53638c971230 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:00.851293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf10c73-4cdd-4e32-970e-53638c971230 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:00.852212 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf10c73-4cdd-4e32-970e-53638c971230 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:00.852750 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf10c73-4cdd-4e32-970e-53638c971230 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:00.854357 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf10c73-4cdd-4e32-970e-53638c971230 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:00.872192 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 372/757] 10.222.0.46 () {56 vars in 1149 bytes} [Fri Sep 9 12:38:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:00.873265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf10c73-4cdd-4e32-970e-53638c971230 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:00.904023 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0807898-dde6-44fd-b43f-d7b0e4afe198 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:00.906416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:00.942549 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b339f-0578-405f-8d58-f9279f7289c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:00.943295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b339f-0578-405f-8d58-f9279f7289c7 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:00.943565 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b339f-0578-405f-8d58-f9279f7289c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:00.943846 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b339f-0578-405f-8d58-f9279f7289c7 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:00.945855 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b339f-0578-405f-8d58-f9279f7289c7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:00.945855 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b339f-0578-405f-8d58-f9279f7289c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:00.947239 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b339f-0578-405f-8d58-f9279f7289c7 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:00.957247 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:00.972191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 386/758] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 12:38:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:00.973339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b339f-0578-405f-8d58-f9279f7289c7 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:00.976866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf10c73-4cdd-4e32-970e-53638c971230 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:01.012398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed9396-a4c8-4f3f-b7a5-b9c73506db9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:01.013347 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed9396-a4c8-4f3f-b7a5-b9c73506db9a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:01.013768 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed9396-a4c8-4f3f-b7a5-b9c73506db9a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:01.014431 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed9396-a4c8-4f3f-b7a5-b9c73506db9a None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:01.015439 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed9396-a4c8-4f3f-b7a5-b9c73506db9a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:01.015878 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed9396-a4c8-4f3f-b7a5-b9c73506db9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:01.017505 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed9396-a4c8-4f3f-b7a5-b9c73506db9a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:01.023856 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 373/759] 10.222.0.46 () {56 vars in 1124 bytes} [Fri Sep 9 12:38:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:01.024854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed9396-a4c8-4f3f-b7a5-b9c73506db9a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:01.108482 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b339f-0578-405f-8d58-f9279f7289c7 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:01.145077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed9396-a4c8-4f3f-b7a5-b9c73506db9a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:01.149388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630a95a-83ff-44da-b18f-8eae1a022b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:01.150074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630a95a-83ff-44da-b18f-8eae1a022b7d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:01.150456 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630a95a-83ff-44da-b18f-8eae1a022b7d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:01.150835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630a95a-83ff-44da-b18f-8eae1a022b7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:01.183331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfd948-db8b-4868-9f51-9e387279d688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:01.184398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfd948-db8b-4868-9f51-9e387279d688 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:01.184900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfd948-db8b-4868-9f51-9e387279d688 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:01.185658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfd948-db8b-4868-9f51-9e387279d688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:01.189794 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:01.197996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfd948-db8b-4868-9f51-9e387279d688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:01.199779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfd948-db8b-4868-9f51-9e387279d688 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:01.208542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-82dfd948-db8b-4868-9f51-9e387279d688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:01.217664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:01.219298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:01.220227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:01.220227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:01.220227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:01.226782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 374/760] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:01.227936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfd948-db8b-4868-9f51-9e387279d688 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:01.233999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630a95a-83ff-44da-b18f-8eae1a022b7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:01.236243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630a95a-83ff-44da-b18f-8eae1a022b7d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:01.352339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfd948-db8b-4868-9f51-9e387279d688 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:01.352859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 387/761] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:38:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:38:01.353962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630a95a-83ff-44da-b18f-8eae1a022b7d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:01.395097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86bddb-7d18-4ade-9048-1e4603dfaded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:01.395778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86bddb-7d18-4ade-9048-1e4603dfaded None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:01.396131 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86bddb-7d18-4ade-9048-1e4603dfaded None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:01.396799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86bddb-7d18-4ade-9048-1e4603dfaded None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:01.397443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86bddb-7d18-4ade-9048-1e4603dfaded None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:01.397744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86bddb-7d18-4ade-9048-1e4603dfaded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:01.399496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86bddb-7d18-4ade-9048-1e4603dfaded None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:01.407117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 375/762] 10.222.0.46 () {56 vars in 1124 bytes} [Fri Sep 9 12:38:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:01.407117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86bddb-7d18-4ade-9048-1e4603dfaded None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:01.424861 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:01.472955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630a95a-83ff-44da-b18f-8eae1a022b7d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:01.485502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:01.516151 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395bd3b-3d33-424d-a22d-a4d53e97272b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:01.517205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395bd3b-3d33-424d-a22d-a4d53e97272b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:01.517595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395bd3b-3d33-424d-a22d-a4d53e97272b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:01.517971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395bd3b-3d33-424d-a22d-a4d53e97272b None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:01.519180 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395bd3b-3d33-424d-a22d-a4d53e97272b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:01.519585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395bd3b-3d33-424d-a22d-a4d53e97272b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:01.521209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395bd3b-3d33-424d-a22d-a4d53e97272b None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:01.527703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86bddb-7d18-4ade-9048-1e4603dfaded None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:01.536158 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 388/763] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:01.538720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395bd3b-3d33-424d-a22d-a4d53e97272b None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:01.570939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ea17b-dbb1-40f1-a6bf-c046dee75226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:01.571564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ea17b-dbb1-40f1-a6bf-c046dee75226 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:01.571842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ea17b-dbb1-40f1-a6bf-c046dee75226 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:01.572111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ea17b-dbb1-40f1-a6bf-c046dee75226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:01.582818 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ea17b-dbb1-40f1-a6bf-c046dee75226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:01.584236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ea17b-dbb1-40f1-a6bf-c046dee75226 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-e74ea17b-dbb1-40f1-a6bf-c046dee75226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:01.598672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:01.599679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:01.600630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:01.600630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:01.600630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:01.600630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 376/764] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:01.600870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ea17b-dbb1-40f1-a6bf-c046dee75226 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:01.654830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395bd3b-3d33-424d-a22d-a4d53e97272b None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:01.678858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:01.691188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bac5ad-9168-4255-ae70-12feb028b401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:01.691797 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bac5ad-9168-4255-ae70-12feb028b401 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:01.693425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bac5ad-9168-4255-ae70-12feb028b401 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:01.693425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bac5ad-9168-4255-ae70-12feb028b401 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:01.693425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bac5ad-9168-4255-ae70-12feb028b401 None admin] RBAC: Adding query filter params (interface=public, service_id=de656787ab794ad497b77f910a0542b0, region_id=RegionOne) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:01.693425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bac5ad-9168-4255-ae70-12feb028b401 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:01.695308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bac5ad-9168-4255-ae70-12feb028b401 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:01.708496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 389/765] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 12:38:01 2022] GET /identity/v3/endpoints?service_id=de656787ab794ad497b77f910a0542b0&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:01.708496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0c7ee-8871-4e2d-819c-0dbafbecd6e7 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:01.708496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0c7ee-8871-4e2d-819c-0dbafbecd6e7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:01.708496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0c7ee-8871-4e2d-819c-0dbafbecd6e7 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:01.708496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 390/766] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:01.708496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0c7ee-8871-4e2d-819c-0dbafbecd6e7 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:01.722437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ea17b-dbb1-40f1-a6bf-c046dee75226 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:01.737015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:01.759662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cf738-c193-4e19-a054-9b718342ed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:01.760298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cf738-c193-4e19-a054-9b718342ed84 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:01.760579 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cf738-c193-4e19-a054-9b718342ed84 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:01.760860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cf738-c193-4e19-a054-9b718342ed84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:01.767842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cf738-c193-4e19-a054-9b718342ed84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:01.769282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cf738-c193-4e19-a054-9b718342ed84 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-1c1cf738-c193-4e19-a054-9b718342ed84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:01.781225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:01.782260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:01.783195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:01.783195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:01.783195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:01.783195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:01.783195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:01.783195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:01.783195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:01.783195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:01.787224 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 377/767] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:01.788092 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cf738-c193-4e19-a054-9b718342ed84 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:01.823037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0c7ee-8871-4e2d-819c-0dbafbecd6e7 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:01.862082 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245937a6-ec1d-4c4b-aca8-76fa4d0c15da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:01.863106 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245937a6-ec1d-4c4b-aca8-76fa4d0c15da None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:01.863583 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245937a6-ec1d-4c4b-aca8-76fa4d0c15da None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:01.864107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245937a6-ec1d-4c4b-aca8-76fa4d0c15da None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:01.864775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245937a6-ec1d-4c4b-aca8-76fa4d0c15da None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:01.865085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245937a6-ec1d-4c4b-aca8-76fa4d0c15da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:01.867010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245937a6-ec1d-4c4b-aca8-76fa4d0c15da None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:01.882893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 391/768] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:01.882893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea3e4f-fb26-43e8-b77a-ff467a62537f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:01.882893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea3e4f-fb26-43e8-b77a-ff467a62537f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:01.882893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea3e4f-fb26-43e8-b77a-ff467a62537f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:01.898256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cf738-c193-4e19-a054-9b718342ed84 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:01.898676 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-c1ea3e4f-fb26-43e8-b77a-ff467a62537f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:01.899867 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c1ea3e4f-fb26-43e8-b77a-ff467a62537f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:01.936148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb98e7a-c3dc-49ff-8eec-02c1ed2f10a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:01.937476 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb98e7a-c3dc-49ff-8eec-02c1ed2f10a6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:01.937756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb98e7a-c3dc-49ff-8eec-02c1ed2f10a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:01.938024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb98e7a-c3dc-49ff-8eec-02c1ed2f10a6 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:01.938650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb98e7a-c3dc-49ff-8eec-02c1ed2f10a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:01.940018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb98e7a-c3dc-49ff-8eec-02c1ed2f10a6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:01.943619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:01.954532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 378/769] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 12:38:01 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:01.954532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb98e7a-c3dc-49ff-8eec-02c1ed2f10a6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:01.969877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 392/770] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:01.970815 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea3e4f-fb26-43e8-b77a-ff467a62537f None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:02.045644 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:02.165743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb98e7a-c3dc-49ff-8eec-02c1ed2f10a6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:02.180891 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea3e4f-fb26-43e8-b77a-ff467a62537f None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:02.208847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145e2d3-b0e5-4c00-8e1d-5dd6c2eaffa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:02.209393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e2d3-b0e5-4c00-8e1d-5dd6c2eaffa7 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:02.209678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e2d3-b0e5-4c00-8e1d-5dd6c2eaffa7 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:02.209987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e2d3-b0e5-4c00-8e1d-5dd6c2eaffa7 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:02.210709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145e2d3-b0e5-4c00-8e1d-5dd6c2eaffa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:02.211016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145e2d3-b0e5-4c00-8e1d-5dd6c2eaffa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:02.212453 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145e2d3-b0e5-4c00-8e1d-5dd6c2eaffa7 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:02.228769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 379/771] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:38:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:02.229981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0a119-2493-4911-b235-9731daf0748a None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:02.230270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0a119-2493-4911-b235-9731daf0748a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:02.230587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0a119-2493-4911-b235-9731daf0748a None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:02.231785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 380/772] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:02.232294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616f57b-d17b-45ac-9c8b-5916607c981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:02.233210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616f57b-d17b-45ac-9c8b-5916607c981c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:02.233680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616f57b-d17b-45ac-9c8b-5916607c981c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:02.234143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616f57b-d17b-45ac-9c8b-5916607c981c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:02.236025 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0a119-2493-4911-b235-9731daf0748a None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:02.244426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616f57b-d17b-45ac-9c8b-5916607c981c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:02.246191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616f57b-d17b-45ac-9c8b-5916607c981c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-2616f57b-d17b-45ac-9c8b-5916607c981c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:02.261942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:02.262926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:02.263821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:02.263821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:02.263821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:02.263821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:02.263821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:02.263821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:02.263821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:02.263821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:02.266222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 393/773] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:02.271775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a21049-df5b-40b7-8402-04e22e446315 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:02.272054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a21049-df5b-40b7-8402-04e22e446315 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:02.272324 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a21049-df5b-40b7-8402-04e22e446315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:02.291582 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b4a21049-df5b-40b7-8402-04e22e446315 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:02.292762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b4a21049-df5b-40b7-8402-04e22e446315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:02.321441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:02.352227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:02.357047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0a119-2493-4911-b235-9731daf0748a None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:02.361228 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 394/774] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:02.362077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a21049-df5b-40b7-8402-04e22e446315 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:02.396372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25254cb9-c75a-43bb-9b8e-aab9e1fe16b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:02.397639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25254cb9-c75a-43bb-9b8e-aab9e1fe16b1 None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:02.398550 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25254cb9-c75a-43bb-9b8e-aab9e1fe16b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:02.399112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25254cb9-c75a-43bb-9b8e-aab9e1fe16b1 None admin] PATH_INFO: `/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/f29af98c5fba4569b16edf240b13c5c5/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:02.401630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25254cb9-c75a-43bb-9b8e-aab9e1fe16b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e2dc716277d42ae8316ebb2d36293a8, user_id=f29af98c5fba4569b16edf240b13c5c5, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:02.403881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25254cb9-c75a-43bb-9b8e-aab9e1fe16b1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:02.474031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a21049-df5b-40b7-8402-04e22e446315 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:02.483658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 381/775] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:38:02 2022] PUT /identity/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/f29af98c5fba4569b16edf240b13c5c5/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:38:02.484834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25254cb9-c75a-43bb-9b8e-aab9e1fe16b1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:02.552010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8d80e-53fa-4200-974d-d123d71d97fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:02.552639 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8d80e-53fa-4200-974d-d123d71d97fa None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:02.552917 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8d80e-53fa-4200-974d-d123d71d97fa None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:02.553187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8d80e-53fa-4200-974d-d123d71d97fa None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:02.554415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8d80e-53fa-4200-974d-d123d71d97fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:02.554741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8d80e-53fa-4200-974d-d123d71d97fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:02.556144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8d80e-53fa-4200-974d-d123d71d97fa None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:02.580865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 395/776] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:38:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:02.588081 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8d80e-53fa-4200-974d-d123d71d97fa None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:02.624798 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25254cb9-c75a-43bb-9b8e-aab9e1fe16b1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:02.650933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:02.676987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653bec5-d723-42c6-9f73-80679dd1cf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:02.677705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653bec5-d723-42c6-9f73-80679dd1cf45 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:02.677983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653bec5-d723-42c6-9f73-80679dd1cf45 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:02.678284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653bec5-d723-42c6-9f73-80679dd1cf45 None admin] PATH_INFO: `/v3/services/network` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:02.680316 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1653bec5-d723-42c6-9f73-80679dd1cf45 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:02.681116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1653bec5-d723-42c6-9f73-80679dd1cf45 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-1653bec5-d723-42c6-9f73-80679dd1cf45 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:02.693001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:02.696937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:02.697797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 382/777] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:02 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:02.698626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653bec5-d723-42c6-9f73-80679dd1cf45 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:02.727084 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:02.758622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8d80e-53fa-4200-974d-d123d71d97fa None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:02.803318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e04755-752a-466c-bc23-604f5671f221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:02.806032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e04755-752a-466c-bc23-604f5671f221 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:02.806555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e04755-752a-466c-bc23-604f5671f221 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:02.807279 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e04755-752a-466c-bc23-604f5671f221 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:02.808506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e04755-752a-466c-bc23-604f5671f221 None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=ec77f4e9939b4dd685b850f8b6fb1e56) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:02.809510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e04755-752a-466c-bc23-604f5671f221 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:02.812018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e04755-752a-466c-bc23-604f5671f221 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:02.818361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653bec5-d723-42c6-9f73-80679dd1cf45 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:02.821122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 396/778] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:38:02 2022] GET /identity/v3/role_assignments?user.id=ec77f4e9939b4dd685b850f8b6fb1e56&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:02.823118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e04755-752a-466c-bc23-604f5671f221 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:02.880116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a2bff-2758-4e8b-b10f-1cdd357bf234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:02.880881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a2bff-2758-4e8b-b10f-1cdd357bf234 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:02.881233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a2bff-2758-4e8b-b10f-1cdd357bf234 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:02.881575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a2bff-2758-4e8b-b10f-1cdd357bf234 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:02.911577 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a2bff-2758-4e8b-b10f-1cdd357bf234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-569a2bff-2758-4e8b-b10f-1cdd357bf234 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:02.918601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:02.919659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:02.920511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:02.920511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:02.920511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:02.920511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:02.920511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:02.920511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:02.920511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:02.920511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:02.920511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:02.920511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:02.920511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:38:02.920511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:02.920511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 383/779] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:38:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:02.926079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a2bff-2758-4e8b-b10f-1cdd357bf234 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:02.974862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:02.989789 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e04755-752a-466c-bc23-604f5671f221 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:03.062997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab7869-59bd-4d7c-b10c-97ef2e62b16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:03.063614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab7869-59bd-4d7c-b10c-97ef2e62b16c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:03.063895 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab7869-59bd-4d7c-b10c-97ef2e62b16c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:03.064168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab7869-59bd-4d7c-b10c-97ef2e62b16c None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:03.064803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab7869-59bd-4d7c-b10c-97ef2e62b16c None admin] RBAC: Adding query filter params (name=network) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:03.065104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab7869-59bd-4d7c-b10c-97ef2e62b16c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:03.066452 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab7869-59bd-4d7c-b10c-97ef2e62b16c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:03.083929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:03.086944 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 397/780] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 12:38:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:03.098018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab7869-59bd-4d7c-b10c-97ef2e62b16c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:03.110183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a2bff-2758-4e8b-b10f-1cdd357bf234 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:03.158682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f1205-790e-49e6-ba9b-91ab036998fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:03.159728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f1205-790e-49e6-ba9b-91ab036998fc None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:03.160242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f1205-790e-49e6-ba9b-91ab036998fc None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:03.160760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f1205-790e-49e6-ba9b-91ab036998fc None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:03.161626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f1205-790e-49e6-ba9b-91ab036998fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:03.162334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f1205-790e-49e6-ba9b-91ab036998fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:03.165756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f1205-790e-49e6-ba9b-91ab036998fc None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:03.180712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 384/781] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:38:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:03.192698 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f1205-790e-49e6-ba9b-91ab036998fc None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:03.254593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab7869-59bd-4d7c-b10c-97ef2e62b16c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:03.289032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988901a0-9582-4162-a41b-3f8841424616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:03.289636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988901a0-9582-4162-a41b-3f8841424616 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:03.289908 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988901a0-9582-4162-a41b-3f8841424616 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:03.290173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988901a0-9582-4162-a41b-3f8841424616 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:03.290890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988901a0-9582-4162-a41b-3f8841424616 None admin] RBAC: Adding query filter params (type=network) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:03.291187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988901a0-9582-4162-a41b-3f8841424616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:03.293004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988901a0-9582-4162-a41b-3f8841424616 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:03.306519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 398/782] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 12:38:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:03.327549 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988901a0-9582-4162-a41b-3f8841424616 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:03.349004 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f1205-790e-49e6-ba9b-91ab036998fc None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:03.386546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eb33a-9e6d-4012-b826-ea95ca59c1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:03.387629 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eb33a-9e6d-4012-b826-ea95ca59c1c7 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:03.388188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eb33a-9e6d-4012-b826-ea95ca59c1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:03.388659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eb33a-9e6d-4012-b826-ea95ca59c1c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:03.391544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:03.401294 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69eb33a-9e6d-4012-b826-ea95ca59c1c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:03.403116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69eb33a-9e6d-4012-b826-ea95ca59c1c7 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:03.410930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-c69eb33a-9e6d-4012-b826-ea95ca59c1c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:03.420253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:03.421237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:03.422585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:03.422585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:03.422585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:03.428461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 385/783] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:03.430443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eb33a-9e6d-4012-b826-ea95ca59c1c7 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:03.490335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988901a0-9582-4162-a41b-3f8841424616 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:03.541046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eb33a-9e6d-4012-b826-ea95ca59c1c7 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:03.571200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2f75b-7661-4d77-8188-b874572618bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:03.571827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2f75b-7661-4d77-8188-b874572618bf None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:03.572101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2f75b-7661-4d77-8188-b874572618bf None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:03.572380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2f75b-7661-4d77-8188-b874572618bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:03.573097 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2f75b-7661-4d77-8188-b874572618bf None admin] RBAC: Adding query filter params (interface=public, service_id=a9869b603b894f48bebb88257148966c, region_id=RegionOne) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:03.573405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2f75b-7661-4d77-8188-b874572618bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:03.574811 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2f75b-7661-4d77-8188-b874572618bf None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:03.583046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf9644-2684-402b-b663-ad340a8a5e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:03.583954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf9644-2684-402b-b663-ad340a8a5e73 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:03.584435 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf9644-2684-402b-b663-ad340a8a5e73 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:03.584933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf9644-2684-402b-b663-ad340a8a5e73 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:03.585897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf9644-2684-402b-b663-ad340a8a5e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:03.586413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf9644-2684-402b-b663-ad340a8a5e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:03.590516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 399/784] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 12:38:03 2022] GET /identity/v3/endpoints?service_id=a9869b603b894f48bebb88257148966c&interface=public®ion_id=RegionOne => generated 194 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:03.592099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf9644-2684-402b-b663-ad340a8a5e73 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:03.610509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2f75b-7661-4d77-8188-b874572618bf None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:03.611187 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 386/785] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:03.612816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf9644-2684-402b-b663-ad340a8a5e73 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:03.695180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:03.720672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf9644-2684-402b-b663-ad340a8a5e73 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:03.793572 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ca386-7825-4af4-9e9a-42550dfd3f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:03.795264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ca386-7825-4af4-9e9a-42550dfd3f14 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:03.797051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ca386-7825-4af4-9e9a-42550dfd3f14 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:03.797803 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ca386-7825-4af4-9e9a-42550dfd3f14 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:03.801062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:03.812073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ca386-7825-4af4-9e9a-42550dfd3f14 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:03.816931 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ca386-7825-4af4-9e9a-42550dfd3f14 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-b25ca386-7825-4af4-9e9a-42550dfd3f14 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:03.842079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:03.843204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:03.844200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:03.844200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:03.844200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:03.844200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:03.844200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:03.844200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:03.844200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:03.844200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:03.844200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:38:03.844200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:03.854266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2f75b-7661-4d77-8188-b874572618bf None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:03.862636 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 387/786] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:38:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:03.864818 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ca386-7825-4af4-9e9a-42550dfd3f14 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:03.933592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2209b-40b1-4b05-b7ad-e84ce26348bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:03.934667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2209b-40b1-4b05-b7ad-e84ce26348bb None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:03.935149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2209b-40b1-4b05-b7ad-e84ce26348bb None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:03.935869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2209b-40b1-4b05-b7ad-e84ce26348bb None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:03.936706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2209b-40b1-4b05-b7ad-e84ce26348bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:03.938729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2209b-40b1-4b05-b7ad-e84ce26348bb None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:03.949888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 400/787] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 12:38:03 2022] GET /identity/v3/services => generated 772 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:04.069966 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ca386-7825-4af4-9e9a-42550dfd3f14 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:04.123265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:04.168978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f5533-25df-4783-a6c2-ff5ee2b18024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:04.169975 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f5533-25df-4783-a6c2-ff5ee2b18024 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:04.170550 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f5533-25df-4783-a6c2-ff5ee2b18024 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:04.171031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f5533-25df-4783-a6c2-ff5ee2b18024 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:04.171989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f5533-25df-4783-a6c2-ff5ee2b18024 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:04.172536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f5533-25df-4783-a6c2-ff5ee2b18024 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:04.174214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f5533-25df-4783-a6c2-ff5ee2b18024 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:04.202597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 388/788] 10.222.0.46 () {56 vars in 1135 bytes} [Fri Sep 9 12:38:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:04.204363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2209b-40b1-4b05-b7ad-e84ce26348bb None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:04.319108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16a503-33e3-4237-9138-a6153d126aad None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:04.319693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16a503-33e3-4237-9138-a6153d126aad None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:04.320301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16a503-33e3-4237-9138-a6153d126aad None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:04.321922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2209b-40b1-4b05-b7ad-e84ce26348bb None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:04.326709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 389/789] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:38:04.336756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41473767-2a43-4744-a8fb-e7ea0fe500f3 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:04.337242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41473767-2a43-4744-a8fb-e7ea0fe500f3 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:04.337646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41473767-2a43-4744-a8fb-e7ea0fe500f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:04.361377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-41473767-2a43-4744-a8fb-e7ea0fe500f3 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:04.362885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-41473767-2a43-4744-a8fb-e7ea0fe500f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:04.375331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8d8df-0157-4367-b2e8-85a2d0a028eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:04.376174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8d8df-0157-4367-b2e8-85a2d0a028eb None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:04.376620 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8d8df-0157-4367-b2e8-85a2d0a028eb None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:04.377068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8d8df-0157-4367-b2e8-85a2d0a028eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:04.394582 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8d8df-0157-4367-b2e8-85a2d0a028eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:04.396961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8d8df-0157-4367-b2e8-85a2d0a028eb None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:04.411943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-f1f8d8df-0157-4367-b2e8-85a2d0a028eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:04.421294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:04.427346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:04.428288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:04.428288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:04.428288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:04.428288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 401/790] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:04.428288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8d8df-0157-4367-b2e8-85a2d0a028eb None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:04.478399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 390/791] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:04.566159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8d8df-0157-4367-b2e8-85a2d0a028eb None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:04.616149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd933d-608e-401d-8dfb-0385488d0cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:04.617113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd933d-608e-401d-8dfb-0385488d0cd2 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:04.617608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd933d-608e-401d-8dfb-0385488d0cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:04.618249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd933d-608e-401d-8dfb-0385488d0cd2 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:04.621749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd933d-608e-401d-8dfb-0385488d0cd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:04.623434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd933d-608e-401d-8dfb-0385488d0cd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:04.625031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd933d-608e-401d-8dfb-0385488d0cd2 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:04.642747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 402/792] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:04.648397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41473767-2a43-4744-a8fb-e7ea0fe500f3 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:04.767517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41473767-2a43-4744-a8fb-e7ea0fe500f3 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:04.769827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:04.802569 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bf66f-a9bc-489f-91c5-05b459c70c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:04.803204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bf66f-a9bc-489f-91c5-05b459c70c0c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:04.803487 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bf66f-a9bc-489f-91c5-05b459c70c0c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:04.804299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bf66f-a9bc-489f-91c5-05b459c70c0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:04.811485 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bf66f-a9bc-489f-91c5-05b459c70c0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:04.813332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bf66f-a9bc-489f-91c5-05b459c70c0c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-459bf66f-a9bc-489f-91c5-05b459c70c0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:04.825172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:04.826095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:04.827410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:04.827410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:04.827410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:04.827410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:04.827410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:04.827410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:04.827410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:04.827410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:04.827410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 391/793] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:04.832217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd933d-608e-401d-8dfb-0385488d0cd2 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:04.846337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bf66f-a9bc-489f-91c5-05b459c70c0c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:04.899218 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:04.945857 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd933d-608e-401d-8dfb-0385488d0cd2 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:04.984079 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4026aab-6a47-44b8-ae73-6b805a04627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:04.985514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4026aab-6a47-44b8-ae73-6b805a04627d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:04.985992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4026aab-6a47-44b8-ae73-6b805a04627d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:04.986529 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4026aab-6a47-44b8-ae73-6b805a04627d None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:04.987480 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4026aab-6a47-44b8-ae73-6b805a04627d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:04.987978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4026aab-6a47-44b8-ae73-6b805a04627d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:04.989641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4026aab-6a47-44b8-ae73-6b805a04627d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:05.003629 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 403/794] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:38:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:05.009796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4026aab-6a47-44b8-ae73-6b805a04627d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:05.113751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bf66f-a9bc-489f-91c5-05b459c70c0c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:05.127902 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4026aab-6a47-44b8-ae73-6b805a04627d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:05.150969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:05.162683 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23a2ba-9878-43c1-ad33-7242fa322183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:05.164730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23a2ba-9878-43c1-ad33-7242fa322183 None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:05.165214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23a2ba-9878-43c1-ad33-7242fa322183 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:05.165790 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23a2ba-9878-43c1-ad33-7242fa322183 None admin] PATH_INFO: `/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/e8e4805f0b1e484c98fb399313f88ef2/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:05.167954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23a2ba-9878-43c1-ad33-7242fa322183 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e2dc716277d42ae8316ebb2d36293a8, user_id=e8e4805f0b1e484c98fb399313f88ef2, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:05.169794 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23a2ba-9878-43c1-ad33-7242fa322183 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:05.199509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba946dd9-cd03-4cd9-976e-73ec8f71a0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:05.200953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba946dd9-cd03-4cd9-976e-73ec8f71a0c6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:05.201462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba946dd9-cd03-4cd9-976e-73ec8f71a0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:05.202067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba946dd9-cd03-4cd9-976e-73ec8f71a0c6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:05.202917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba946dd9-cd03-4cd9-976e-73ec8f71a0c6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:05.204522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba946dd9-cd03-4cd9-976e-73ec8f71a0c6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-ba946dd9-cd03-4cd9-976e-73ec8f71a0c6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:05.215662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:05.216727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:38:05.217602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:05.220456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 392/795] 10.222.0.46 () {56 vars in 1129 bytes} [Fri Sep 9 12:38:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:05.227090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba946dd9-cd03-4cd9-976e-73ec8f71a0c6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:05.259399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:05.287033 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 404/796] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:38:05 2022] PUT /identity/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/e8e4805f0b1e484c98fb399313f88ef2/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:38:05.356432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba946dd9-cd03-4cd9-976e-73ec8f71a0c6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:05.398298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585ea49-103a-4e90-b191-fc979aa7cb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:05.399415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585ea49-103a-4e90-b191-fc979aa7cb8f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:05.399958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585ea49-103a-4e90-b191-fc979aa7cb8f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:05.400471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585ea49-103a-4e90-b191-fc979aa7cb8f None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:05.401401 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585ea49-103a-4e90-b191-fc979aa7cb8f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:05.401918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585ea49-103a-4e90-b191-fc979aa7cb8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:05.403592 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585ea49-103a-4e90-b191-fc979aa7cb8f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:05.408090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7672ea-4e74-4ec9-bf6f-a3fe1bca3f0f None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:05.408381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7672ea-4e74-4ec9-bf6f-a3fe1bca3f0f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:05.408648 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7672ea-4e74-4ec9-bf6f-a3fe1bca3f0f None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:05.410315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 393/798] 10.222.0.46 () {56 vars in 1124 bytes} [Fri Sep 9 12:38:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:05.410427 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 405/798] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:05.416209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0634a2-9753-4eb2-9197-baa4293644a6 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:05.416473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585ea49-103a-4e90-b191-fc979aa7cb8f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:05.416579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0634a2-9753-4eb2-9197-baa4293644a6 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:05.416753 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0634a2-9753-4eb2-9197-baa4293644a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:05.467677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-3c0634a2-9753-4eb2-9197-baa4293644a6 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:05.468912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-3c0634a2-9753-4eb2-9197-baa4293644a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:05.522189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:05.532409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585ea49-103a-4e90-b191-fc979aa7cb8f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:05.569278 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 406/799] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:05.571284 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560f341-a8e5-49d7-857b-2db9b227192c None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:05.571565 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560f341-a8e5-49d7-857b-2db9b227192c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:05.571833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560f341-a8e5-49d7-857b-2db9b227192c None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:05.573207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 407/800] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:05.576594 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4773ff4-5161-45c8-84f4-c707bf77f8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:05.577768 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4773ff4-5161-45c8-84f4-c707bf77f8e0 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:05.578113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4773ff4-5161-45c8-84f4-c707bf77f8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:05.578387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4773ff4-5161-45c8-84f4-c707bf77f8e0 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:05.579052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4773ff4-5161-45c8-84f4-c707bf77f8e0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:05.579250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d948d21-73bc-4b76-bbe6-48fc3fff872c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:05.579334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d948d21-73bc-4b76-bbe6-48fc3fff872c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:05.579405 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4773ff4-5161-45c8-84f4-c707bf77f8e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:05.579597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d948d21-73bc-4b76-bbe6-48fc3fff872c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:05.580706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4773ff4-5161-45c8-84f4-c707bf77f8e0 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:05.586563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 394/801] 10.222.0.46 () {56 vars in 1124 bytes} [Fri Sep 9 12:38:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:05.596156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4773ff4-5161-45c8-84f4-c707bf77f8e0 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:05.601996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-3d948d21-73bc-4b76-bbe6-48fc3fff872c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:05.603200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-3d948d21-73bc-4b76-bbe6-48fc3fff872c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:05.678717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 408/802] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:05.717042 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4773ff4-5161-45c8-84f4-c707bf77f8e0 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:05.753259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5daaf-4390-46f6-b7e9-3d554387cf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:05.753865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5daaf-4390-46f6-b7e9-3d554387cf97 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:05.754147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5daaf-4390-46f6-b7e9-3d554387cf97 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:05.754421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5daaf-4390-46f6-b7e9-3d554387cf97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:05.755121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5daaf-4390-46f6-b7e9-3d554387cf97 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:05.756481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5daaf-4390-46f6-b7e9-3d554387cf97 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:05.846760 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d948d21-73bc-4b76-bbe6-48fc3fff872c None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:05.883948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 395/803] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:38:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:38:05.891167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:05.973026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d948d21-73bc-4b76-bbe6-48fc3fff872c None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:05.976007 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5daaf-4390-46f6-b7e9-3d554387cf97 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:06.058412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0384cf3-e456-4bf9-9fa4-29b170de2320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:06.059485 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0384cf3-e456-4bf9-9fa4-29b170de2320 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:06.060000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0384cf3-e456-4bf9-9fa4-29b170de2320 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:06.060764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0384cf3-e456-4bf9-9fa4-29b170de2320 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:06.079456 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0384cf3-e456-4bf9-9fa4-29b170de2320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-f0384cf3-e456-4bf9-9fa4-29b170de2320 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:06.085312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:06.086745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:06.087622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:06.087622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:06.087622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:06.087622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:06.087622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:06.087622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:06.087622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:06.087622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:06.087622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:06.087622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:06.087622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:38:06.087622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:06.094516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 409/804] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:38:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:06.097496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0384cf3-e456-4bf9-9fa4-29b170de2320 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:06.118046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:06.231256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5daaf-4390-46f6-b7e9-3d554387cf97 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:06.241059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:06.250841 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0384cf3-e456-4bf9-9fa4-29b170de2320 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:06.270721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e842d-9be3-4759-8df6-e53db331d29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:06.271460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e842d-9be3-4759-8df6-e53db331d29b None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:06.271809 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e842d-9be3-4759-8df6-e53db331d29b None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:06.272171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e842d-9be3-4759-8df6-e53db331d29b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:06.291144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e842d-9be3-4759-8df6-e53db331d29b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-8f9e842d-9be3-4759-8df6-e53db331d29b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:06.298024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:06.299046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:06.299907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:06.299907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:06.299907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:06.299907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:06.299907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:06.299907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:06.299907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:06.299907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:06.299907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:06.299907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:06.299907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:38:06.299907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:06.299907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 396/805] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:38:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:06.306227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e842d-9be3-4759-8df6-e53db331d29b None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:06.310710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6072251-efbf-4eba-ba03-95abef16bb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:06.311334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6072251-efbf-4eba-ba03-95abef16bb4c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:06.311607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6072251-efbf-4eba-ba03-95abef16bb4c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:06.311876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6072251-efbf-4eba-ba03-95abef16bb4c None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:06.312550 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6072251-efbf-4eba-ba03-95abef16bb4c None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:06.312849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6072251-efbf-4eba-ba03-95abef16bb4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:06.314515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6072251-efbf-4eba-ba03-95abef16bb4c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:06.337536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 410/806] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:38:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:06.347521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6072251-efbf-4eba-ba03-95abef16bb4c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:06.391000 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:06.453443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e842d-9be3-4759-8df6-e53db331d29b None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:06.498253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6072251-efbf-4eba-ba03-95abef16bb4c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:06.545077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ec0c8-486a-40a0-813a-4c6670c9e993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:06.546743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ec0c8-486a-40a0-813a-4c6670c9e993 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:06.547557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ec0c8-486a-40a0-813a-4c6670c9e993 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:06.548358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca223a-37d6-4bcf-b85c-5986ba2aa5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:06.548850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ec0c8-486a-40a0-813a-4c6670c9e993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:06.550050 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca223a-37d6-4bcf-b85c-5986ba2aa5a7 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:06.550792 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca223a-37d6-4bcf-b85c-5986ba2aa5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:06.552475 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca223a-37d6-4bcf-b85c-5986ba2aa5a7 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:06.553357 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca223a-37d6-4bcf-b85c-5986ba2aa5a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:06.554888 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca223a-37d6-4bcf-b85c-5986ba2aa5a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:06.556531 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca223a-37d6-4bcf-b85c-5986ba2aa5a7 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:06.573479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ec0c8-486a-40a0-813a-4c6670c9e993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:06.577378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ec0c8-486a-40a0-813a-4c6670c9e993 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:06.577855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 397/807] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:38:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:06.581619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca223a-37d6-4bcf-b85c-5986ba2aa5a7 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-706ec0c8-486a-40a0-813a-4c6670c9e993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:06.606668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:06.607671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:06.608602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:06.608602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:06.608602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:06.615135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 411/808] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:06.620900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ec0c8-486a-40a0-813a-4c6670c9e993 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:06.647090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:06.738613 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca223a-37d6-4bcf-b85c-5986ba2aa5a7 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:06.779603 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d6417-2838-407a-b5e7-7d75dbf75f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:06.780296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d6417-2838-407a-b5e7-7d75dbf75f4f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:06.780581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d6417-2838-407a-b5e7-7d75dbf75f4f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:06.780854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d6417-2838-407a-b5e7-7d75dbf75f4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:06.783786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:06.794501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d6417-2838-407a-b5e7-7d75dbf75f4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:06.796215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d6417-2838-407a-b5e7-7d75dbf75f4f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-311d6417-2838-407a-b5e7-7d75dbf75f4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:06.815156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:06.816321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:06.817386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:06.817386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:06.817386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:06.820581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ec0c8-486a-40a0-813a-4c6670c9e993 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:06.823558 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 398/809] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:06.824897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb817697-9133-4e0d-8cef-8712cd4ff7b9 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:06.825185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb817697-9133-4e0d-8cef-8712cd4ff7b9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:06.825461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb817697-9133-4e0d-8cef-8712cd4ff7b9 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:06.827035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 399/810] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:06.827730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb817697-9133-4e0d-8cef-8712cd4ff7b9 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:06.873090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae5fe2-6238-44de-8f4f-23283bfa6640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:06.873759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae5fe2-6238-44de-8f4f-23283bfa6640 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:06.873962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae5fe2-6238-44de-8f4f-23283bfa6640 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:06.874495 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae5fe2-6238-44de-8f4f-23283bfa6640 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:06.875214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae5fe2-6238-44de-8f4f-23283bfa6640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:06.875581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae5fe2-6238-44de-8f4f-23283bfa6640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:06.877005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae5fe2-6238-44de-8f4f-23283bfa6640 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:06.949331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb817697-9133-4e0d-8cef-8712cd4ff7b9 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:06.988267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728d31f-89cc-497d-8934-7bb6ad142934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:06.988901 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728d31f-89cc-497d-8934-7bb6ad142934 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:06.989168 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728d31f-89cc-497d-8934-7bb6ad142934 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:06.989450 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728d31f-89cc-497d-8934-7bb6ad142934 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:06.990092 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728d31f-89cc-497d-8934-7bb6ad142934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:06.990398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728d31f-89cc-497d-8934-7bb6ad142934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:06.992812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728d31f-89cc-497d-8934-7bb6ad142934 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:07.006972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 400/811] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:07.008195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce92212-af9a-4dcf-a292-3384036dec0d None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:07.008481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce92212-af9a-4dcf-a292-3384036dec0d None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:07.008587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 412/812] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:07.008755 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce92212-af9a-4dcf-a292-3384036dec0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:07.016382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae5fe2-6238-44de-8f4f-23283bfa6640 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:07.031428 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-5ce92212-af9a-4dcf-a292-3384036dec0d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:07.032978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5ce92212-af9a-4dcf-a292-3384036dec0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:07.089954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:07.090270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:07.122707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 401/813] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:07.125620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce92212-af9a-4dcf-a292-3384036dec0d None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:07.134036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae5fe2-6238-44de-8f4f-23283bfa6640 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:07.196439 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ca94a-2a39-4905-8c04-ead1401571cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:07.197114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ca94a-2a39-4905-8c04-ead1401571cf None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:07.197399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ca94a-2a39-4905-8c04-ead1401571cf None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:07.197661 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ca94a-2a39-4905-8c04-ead1401571cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:07.209784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ca94a-2a39-4905-8c04-ead1401571cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:07.212761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ca94a-2a39-4905-8c04-ead1401571cf None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-a33ca94a-2a39-4905-8c04-ead1401571cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:07.234708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:07.235681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:07.236630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:07.236630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:07.236630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:07.236630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:07.236630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:07.236630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:07.236630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:07.236630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:07.236630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:38:07.236630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:07.237656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 413/814] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 12:38:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:07.243219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ca94a-2a39-4905-8c04-ead1401571cf None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:07.311157 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce92212-af9a-4dcf-a292-3384036dec0d None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:07.368395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea4845-1847-49e0-89c1-703a7f9c0049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:07.369045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea4845-1847-49e0-89c1-703a7f9c0049 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:07.369322 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea4845-1847-49e0-89c1-703a7f9c0049 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:07.369737 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea4845-1847-49e0-89c1-703a7f9c0049 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:07.385195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea4845-1847-49e0-89c1-703a7f9c0049 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:07.387469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea4845-1847-49e0-89c1-703a7f9c0049 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-0eea4845-1847-49e0-89c1-703a7f9c0049 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:07.410296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:07.412539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:07.415664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:07.415664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:07.415664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:07.415664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:07.415664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:07.415664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:07.415664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:07.415664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:07.415664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:38:07.415664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:07.415664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 402/815] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 12:38:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:07.420018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea4845-1847-49e0-89c1-703a7f9c0049 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:07.437241 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ca94a-2a39-4905-8c04-ead1401571cf None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:07.474581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:07.487697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:07.491444 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab86c2b-6deb-48d3-95b2-80ff66173f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:07.491444 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab86c2b-6deb-48d3-95b2-80ff66173f6e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:07.491444 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab86c2b-6deb-48d3-95b2-80ff66173f6e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:07.491444 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab86c2b-6deb-48d3-95b2-80ff66173f6e None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:07.491931 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab86c2b-6deb-48d3-95b2-80ff66173f6e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:07.491931 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab86c2b-6deb-48d3-95b2-80ff66173f6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:07.492510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab86c2b-6deb-48d3-95b2-80ff66173f6e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:07.517475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 414/816] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 12:38:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:07.518405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab86c2b-6deb-48d3-95b2-80ff66173f6e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:07.565254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea4845-1847-49e0-89c1-703a7f9c0049 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:07.603277 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cabca2-39b5-4671-9c25-9efd1f805054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:07.604096 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cabca2-39b5-4671-9c25-9efd1f805054 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:07.604441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cabca2-39b5-4671-9c25-9efd1f805054 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:07.604778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cabca2-39b5-4671-9c25-9efd1f805054 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:07.605578 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cabca2-39b5-4671-9c25-9efd1f805054 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:07.605972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cabca2-39b5-4671-9c25-9efd1f805054 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:07.607591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cabca2-39b5-4671-9c25-9efd1f805054 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:07.630420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 403/817] 10.222.0.46 () {56 vars in 1149 bytes} [Fri Sep 9 12:38:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:07.631292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cabca2-39b5-4671-9c25-9efd1f805054 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:07.653155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab86c2b-6deb-48d3-95b2-80ff66173f6e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:07.699246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba7984-139a-47bd-82ee-ebb9d5cb971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:07.699246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba7984-139a-47bd-82ee-ebb9d5cb971f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:07.699246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba7984-139a-47bd-82ee-ebb9d5cb971f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:07.699750 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba7984-139a-47bd-82ee-ebb9d5cb971f None admin] PATH_INFO: `/v3/services/network` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:07.700100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba7984-139a-47bd-82ee-ebb9d5cb971f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:07.703287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba7984-139a-47bd-82ee-ebb9d5cb971f None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-19ba7984-139a-47bd-82ee-ebb9d5cb971f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:07.717775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:07.718880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:07.719758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 415/818] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:07 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:07.720608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba7984-139a-47bd-82ee-ebb9d5cb971f None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:07.749332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:07.757799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cabca2-39b5-4671-9c25-9efd1f805054 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:07.786083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:07.808983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056106cc-35f6-4a7b-b1b6-f74593579cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:07.810208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056106cc-35f6-4a7b-b1b6-f74593579cf1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:07.811946 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056106cc-35f6-4a7b-b1b6-f74593579cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:07.812454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056106cc-35f6-4a7b-b1b6-f74593579cf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:07.847838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba7984-139a-47bd-82ee-ebb9d5cb971f None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:07.884894 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee24fc5-7fad-471b-8633-7a35a6d85f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:07.885554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee24fc5-7fad-471b-8633-7a35a6d85f71 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:07.885783 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee24fc5-7fad-471b-8633-7a35a6d85f71 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:07.886035 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee24fc5-7fad-471b-8633-7a35a6d85f71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:07.898107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee24fc5-7fad-471b-8633-7a35a6d85f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:07.899770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee24fc5-7fad-471b-8633-7a35a6d85f71 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:07.914352 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056106cc-35f6-4a7b-b1b6-f74593579cf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-fee24fc5-7fad-471b-8633-7a35a6d85f71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:07.914934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:07.915878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:07.916788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:07.916788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:07.916788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:07.916788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 416/819] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:07.917564 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee24fc5-7fad-471b-8633-7a35a6d85f71 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:07.918946 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056106cc-35f6-4a7b-b1b6-f74593579cf1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:08.016225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:08.029108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 404/820] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:38:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:38:08.029108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e919d51-c97b-4921-b8e9-1e7020a3c2fe None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:08.029108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e919d51-c97b-4921-b8e9-1e7020a3c2fe None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:08.029108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e919d51-c97b-4921-b8e9-1e7020a3c2fe None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:08.029913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 405/821] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:08.031140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e919d51-c97b-4921-b8e9-1e7020a3c2fe None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:08.034773 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee24fc5-7fad-471b-8633-7a35a6d85f71 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:08.036623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:08.078175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b7455-59c6-41f3-b2b0-ff75bc8a1d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:08.079105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b7455-59c6-41f3-b2b0-ff75bc8a1d1a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:08.079527 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b7455-59c6-41f3-b2b0-ff75bc8a1d1a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:08.079936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b7455-59c6-41f3-b2b0-ff75bc8a1d1a None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:08.080905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b7455-59c6-41f3-b2b0-ff75bc8a1d1a None admin] RBAC: Adding query filter params (name=network) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:08.081368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b7455-59c6-41f3-b2b0-ff75bc8a1d1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:08.083553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b7455-59c6-41f3-b2b0-ff75bc8a1d1a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:08.091019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 417/822] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 12:38:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:08.092292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b7455-59c6-41f3-b2b0-ff75bc8a1d1a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:08.163209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e919d51-c97b-4921-b8e9-1e7020a3c2fe None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:08.202176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce97873-34fe-4c0a-b839-4ba282ed442b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:08.202814 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce97873-34fe-4c0a-b839-4ba282ed442b None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:08.203113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce97873-34fe-4c0a-b839-4ba282ed442b None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:08.203359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce97873-34fe-4c0a-b839-4ba282ed442b None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:08.203992 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce97873-34fe-4c0a-b839-4ba282ed442b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:08.204290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce97873-34fe-4c0a-b839-4ba282ed442b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:08.205632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce97873-34fe-4c0a-b839-4ba282ed442b None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:08.213248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b7455-59c6-41f3-b2b0-ff75bc8a1d1a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:08.218870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 406/823] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:08.220125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea3f86-1e38-4330-ad7c-b4a8aa5025ae None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:08.220412 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea3f86-1e38-4330-ad7c-b4a8aa5025ae None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:08.220690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea3f86-1e38-4330-ad7c-b4a8aa5025ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:08.239370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-21ea3f86-1e38-4330-ad7c-b4a8aa5025ae None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:08.240542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-21ea3f86-1e38-4330-ad7c-b4a8aa5025ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:08.252966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20aaae-9b54-409c-a5da-2b11470a214e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:08.253603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20aaae-9b54-409c-a5da-2b11470a214e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:08.253880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20aaae-9b54-409c-a5da-2b11470a214e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:08.254143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20aaae-9b54-409c-a5da-2b11470a214e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:08.265792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20aaae-9b54-409c-a5da-2b11470a214e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:08.267275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20aaae-9b54-409c-a5da-2b11470a214e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-bb20aaae-9b54-409c-a5da-2b11470a214e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:08.281964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:08.282942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:08.283851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:08.283851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:08.283851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:08.288116 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 418/824] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:08.288783 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20aaae-9b54-409c-a5da-2b11470a214e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:08.295496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:08.310684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:08.318722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 407/825] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:08.318722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea3f86-1e38-4330-ad7c-b4a8aa5025ae None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:08.420980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20aaae-9b54-409c-a5da-2b11470a214e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:08.441724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea3f86-1e38-4330-ad7c-b4a8aa5025ae None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:08.469542 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86005cee-26dc-4284-b74f-a5af65d78b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:08.470738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86005cee-26dc-4284-b74f-a5af65d78b8f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:08.471370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86005cee-26dc-4284-b74f-a5af65d78b8f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:08.471838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86005cee-26dc-4284-b74f-a5af65d78b8f None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:08.473187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86005cee-26dc-4284-b74f-a5af65d78b8f None admin] RBAC: Adding query filter params (type=network) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:08.473708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86005cee-26dc-4284-b74f-a5af65d78b8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:08.475261 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86005cee-26dc-4284-b74f-a5af65d78b8f None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:08.477377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202a907-286a-4f50-ad2f-95afa16e237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:08.477998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202a907-286a-4f50-ad2f-95afa16e237d None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:08.478278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202a907-286a-4f50-ad2f-95afa16e237d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:08.478578 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202a907-286a-4f50-ad2f-95afa16e237d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:08.482075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 419/826] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 12:38:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:08.483066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86005cee-26dc-4284-b74f-a5af65d78b8f None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:08.485293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202a907-286a-4f50-ad2f-95afa16e237d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:08.487099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202a907-286a-4f50-ad2f-95afa16e237d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-4202a907-286a-4f50-ad2f-95afa16e237d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:08.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:08.502817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:08.503722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:08.503722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:08.503722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:08.503722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:08.503722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:08.503722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:08.503722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:08.503722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:08.503722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 408/827] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:08.504215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7f8ca-8b7c-419d-b839-1ad5d115b039 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:08.504873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7f8ca-8b7c-419d-b839-1ad5d115b039 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:08.504969 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7f8ca-8b7c-419d-b839-1ad5d115b039 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:08.510645 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 409/828] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:08.510645 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7f8ca-8b7c-419d-b839-1ad5d115b039 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:08.562516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:08.583491 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:08.602204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86005cee-26dc-4284-b74f-a5af65d78b8f None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:08.623386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7f8ca-8b7c-419d-b839-1ad5d115b039 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:08.635245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5a073-74d7-4bbb-9099-973a4e6c6dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:08.635837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5a073-74d7-4bbb-9099-973a4e6c6dd2 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:08.636115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5a073-74d7-4bbb-9099-973a4e6c6dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:08.636378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5a073-74d7-4bbb-9099-973a4e6c6dd2 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:08.637011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5a073-74d7-4bbb-9099-973a4e6c6dd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:08.637303 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5a073-74d7-4bbb-9099-973a4e6c6dd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:08.638672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5a073-74d7-4bbb-9099-973a4e6c6dd2 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:08.650171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 420/829] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:08.650998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5a073-74d7-4bbb-9099-973a4e6c6dd2 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:08.656740 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9159c-d858-4825-9ced-5030bb1fe00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:08.657403 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9159c-d858-4825-9ced-5030bb1fe00d None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:08.657686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9159c-d858-4825-9ced-5030bb1fe00d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:08.657956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9159c-d858-4825-9ced-5030bb1fe00d None admin] PATH_INFO: `/v3/endpoints` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:08.658646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9159c-d858-4825-9ced-5030bb1fe00d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:08.659999 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9159c-d858-4825-9ced-5030bb1fe00d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:08.737254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 410/830] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:38:08 2022] POST /identity/v3/endpoints => generated 337 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:38:08.738648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691aa2f-079e-4845-9ff0-500f039f3f9e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:08.738938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691aa2f-079e-4845-9ff0-500f039f3f9e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:08.739210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691aa2f-079e-4845-9ff0-500f039f3f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:08.760938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-a691aa2f-079e-4845-9ff0-500f039f3f9e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:08.762959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a691aa2f-079e-4845-9ff0-500f039f3f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:08.765561 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5a073-74d7-4bbb-9099-973a4e6c6dd2 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:08.812352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8fabf-9510-4f5d-b277-5926a3a18d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:08.812974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8fabf-9510-4f5d-b277-5926a3a18d9e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:08.813257 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8fabf-9510-4f5d-b277-5926a3a18d9e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:08.813516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8fabf-9510-4f5d-b277-5926a3a18d9e None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:08.814224 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8fabf-9510-4f5d-b277-5926a3a18d9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:08.814544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8fabf-9510-4f5d-b277-5926a3a18d9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:08.815997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8fabf-9510-4f5d-b277-5926a3a18d9e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:08.823817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:08.829221 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 421/831] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:08.830075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8fabf-9510-4f5d-b277-5926a3a18d9e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:08.852215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 411/832] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:08.853370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691aa2f-079e-4845-9ff0-500f039f3f9e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:08.900421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:08.988942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8fabf-9510-4f5d-b277-5926a3a18d9e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:09.014560 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691aa2f-079e-4845-9ff0-500f039f3f9e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:09.033610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9671604-b598-4708-912d-1388a57722ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:09.034381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9671604-b598-4708-912d-1388a57722ef None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:09.035006 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9671604-b598-4708-912d-1388a57722ef None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:09.035116 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9671604-b598-4708-912d-1388a57722ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:09.048550 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9671604-b598-4708-912d-1388a57722ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-c9671604-b598-4708-912d-1388a57722ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:09.054877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:09.056050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:09.056930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:09.056930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:09.056930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:09.056930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:09.056930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:09.056930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:09.056930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:09.056930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:09.056930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:09.056930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:09.056930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:38:09.056930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:09.056930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 422/833] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:38:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:09.058226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9671604-b598-4708-912d-1388a57722ef None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:09.066031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b4777-2836-440f-898e-897e837d4e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:09.067203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b4777-2836-440f-898e-897e837d4e0e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:09.067698 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b4777-2836-440f-898e-897e837d4e0e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:09.067987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b4777-2836-440f-898e-897e837d4e0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:09.076297 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b4777-2836-440f-898e-897e837d4e0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:09.077742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b4777-2836-440f-898e-897e837d4e0e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-fe4b4777-2836-440f-898e-897e837d4e0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:09.090961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:09.092303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:09.094071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:09.094071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:09.094071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:09.094071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:09.094071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:09.094071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:09.094071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:09.094071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:09.094071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 412/834] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:09.094071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b4777-2836-440f-898e-897e837d4e0e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:09.141352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:09.191464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9671604-b598-4708-912d-1388a57722ef None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:09.211871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:09.220880 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b4777-2836-440f-898e-897e837d4e0e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:09.225918 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c79340-040b-4ce0-a5b5-8895497e9bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:09.226611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c79340-040b-4ce0-a5b5-8895497e9bc2 None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:09.227028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c79340-040b-4ce0-a5b5-8895497e9bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:09.227296 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c79340-040b-4ce0-a5b5-8895497e9bc2 None admin] PATH_INFO: `/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/ec77f4e9939b4dd685b850f8b6fb1e56/roles/4a5e4b92c86041c6aa22c4f4735b4a05` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:09.243611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c79340-040b-4ce0-a5b5-8895497e9bc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e2dc716277d42ae8316ebb2d36293a8, user_id=ec77f4e9939b4dd685b850f8b6fb1e56, role_id=4a5e4b92c86041c6aa22c4f4735b4a05)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:09.244956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c79340-040b-4ce0-a5b5-8895497e9bc2 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:09.262694 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a41ba-6893-4d8b-af71-f98e448b78c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:09.263310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a41ba-6893-4d8b-af71-f98e448b78c4 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:09.266299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a41ba-6893-4d8b-af71-f98e448b78c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:09.266299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a41ba-6893-4d8b-af71-f98e448b78c4 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:09.266299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a41ba-6893-4d8b-af71-f98e448b78c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:09.266299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a41ba-6893-4d8b-af71-f98e448b78c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:09.266299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a41ba-6893-4d8b-af71-f98e448b78c4 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:09.275248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 413/835] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:38:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:09.276156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a41ba-6893-4d8b-af71-f98e448b78c4 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:09.333175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 423/836] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:38:09 2022] PUT /identity/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/ec77f4e9939b4dd685b850f8b6fb1e56/roles/4a5e4b92c86041c6aa22c4f4735b4a05 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:38:09.334258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c79340-040b-4ce0-a5b5-8895497e9bc2 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:09.395584 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a41ba-6893-4d8b-af71-f98e448b78c4 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:09.440694 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7252a1b-0bab-4662-8fef-0505f0f59720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:09.441319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7252a1b-0bab-4662-8fef-0505f0f59720 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:09.441601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7252a1b-0bab-4662-8fef-0505f0f59720 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:09.441875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7252a1b-0bab-4662-8fef-0505f0f59720 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:09.442593 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7252a1b-0bab-4662-8fef-0505f0f59720 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:09.442904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7252a1b-0bab-4662-8fef-0505f0f59720 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:09.444395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7252a1b-0bab-4662-8fef-0505f0f59720 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:09.457850 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 414/837] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:38:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:09.459071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7252a1b-0bab-4662-8fef-0505f0f59720 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:09.482303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:09.498576 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c79340-040b-4ce0-a5b5-8895497e9bc2 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:09.509893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:09.543701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18150e-1752-4878-8d71-6afc3f5a6876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:09.544708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18150e-1752-4878-8d71-6afc3f5a6876 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:09.544708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18150e-1752-4878-8d71-6afc3f5a6876 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:09.544955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18150e-1752-4878-8d71-6afc3f5a6876 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:09.546134 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18150e-1752-4878-8d71-6afc3f5a6876 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:09.547435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18150e-1752-4878-8d71-6afc3f5a6876 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-ed18150e-1752-4878-8d71-6afc3f5a6876 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:09.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:09.562390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:38:09.563288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:09.566927 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 424/838] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:38:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:09.568308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18150e-1752-4878-8d71-6afc3f5a6876 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:09.593930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7252a1b-0bab-4662-8fef-0505f0f59720 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:09.633948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4e8c3-aafd-46e2-976d-37c65c9ee93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:09.634743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4e8c3-aafd-46e2-976d-37c65c9ee93f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:09.635086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4e8c3-aafd-46e2-976d-37c65c9ee93f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:09.635722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4e8c3-aafd-46e2-976d-37c65c9ee93f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:09.647402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4e8c3-aafd-46e2-976d-37c65c9ee93f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:09.649755 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4e8c3-aafd-46e2-976d-37c65c9ee93f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-2fc4e8c3-aafd-46e2-976d-37c65c9ee93f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:09.665660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:09.666928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:09.668748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:09.668748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:09.668748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:09.671110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 415/839] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:09.671110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4e8c3-aafd-46e2-976d-37c65c9ee93f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:09.700034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18150e-1752-4878-8d71-6afc3f5a6876 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:09.737782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87e88b-e7e6-414d-b170-fd3114dcbe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:09.738467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87e88b-e7e6-414d-b170-fd3114dcbe26 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:09.738772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87e88b-e7e6-414d-b170-fd3114dcbe26 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:09.739045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87e88b-e7e6-414d-b170-fd3114dcbe26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:09.739833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87e88b-e7e6-414d-b170-fd3114dcbe26 None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=f29af98c5fba4569b16edf240b13c5c5) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:09.740133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87e88b-e7e6-414d-b170-fd3114dcbe26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:09.741720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87e88b-e7e6-414d-b170-fd3114dcbe26 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:09.758602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 425/840] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:38:09 2022] GET /identity/v3/role_assignments?user.id=f29af98c5fba4569b16edf240b13c5c5&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:09.758602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87e88b-e7e6-414d-b170-fd3114dcbe26 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:09.763268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:09.801382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:09.822622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4e8c3-aafd-46e2-976d-37c65c9ee93f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:09.864920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f07183-b847-431e-a5a5-6622386fd2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:09.865803 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f07183-b847-431e-a5a5-6622386fd2aa None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:09.866327 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f07183-b847-431e-a5a5-6622386fd2aa None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:09.866727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f07183-b847-431e-a5a5-6622386fd2aa None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:09.867974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f07183-b847-431e-a5a5-6622386fd2aa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:09.868384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f07183-b847-431e-a5a5-6622386fd2aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:09.869912 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f07183-b847-431e-a5a5-6622386fd2aa None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:09.880518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 416/841] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:38:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:09.882897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f07183-b847-431e-a5a5-6622386fd2aa None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:09.909607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87e88b-e7e6-414d-b170-fd3114dcbe26 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:09.973609 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131613e-537c-481f-ab78-300b442bae75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:09.974511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131613e-537c-481f-ab78-300b442bae75 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:09.974790 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131613e-537c-481f-ab78-300b442bae75 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:09.975060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131613e-537c-481f-ab78-300b442bae75 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:09.975807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131613e-537c-481f-ab78-300b442bae75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:09.976454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131613e-537c-481f-ab78-300b442bae75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:09.978544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131613e-537c-481f-ab78-300b442bae75 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:09.998292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 426/842] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:10.003952 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131613e-537c-481f-ab78-300b442bae75 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:10.030396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f07183-b847-431e-a5a5-6622386fd2aa None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:10.067836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:10.078274 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08545ff-a169-4218-af7b-6e1caaf33aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:10.079640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08545ff-a169-4218-af7b-6e1caaf33aa8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:10.085462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08545ff-a169-4218-af7b-6e1caaf33aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:10.085462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08545ff-a169-4218-af7b-6e1caaf33aa8 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:10.085462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08545ff-a169-4218-af7b-6e1caaf33aa8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:10.085462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08545ff-a169-4218-af7b-6e1caaf33aa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:10.085462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08545ff-a169-4218-af7b-6e1caaf33aa8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:10.090700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 417/843] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:38:09 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:10.125349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:10.132372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131613e-537c-481f-ab78-300b442bae75 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:10.172456 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926b4c7-68e7-4401-b2f4-681c9c09ff7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:10.173267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926b4c7-68e7-4401-b2f4-681c9c09ff7a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:10.173933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926b4c7-68e7-4401-b2f4-681c9c09ff7a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:10.174270 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926b4c7-68e7-4401-b2f4-681c9c09ff7a None admin] PATH_INFO: `/v3/users/swift` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:10.183712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926b4c7-68e7-4401-b2f4-681c9c09ff7a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:10.185247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926b4c7-68e7-4401-b2f4-681c9c09ff7a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-3926b4c7-68e7-4401-b2f4-681c9c09ff7a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:10.199142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:10.200160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:10.201102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:10.201102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:10.201102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:10.201102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:10.201102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:10.201102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:10.201102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:10.201102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:10.201102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:38:10.201102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:10.201102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 427/844] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:38:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:10.206450 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08545ff-a169-4218-af7b-6e1caaf33aa8 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:10.328877 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08545ff-a169-4218-af7b-6e1caaf33aa8 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:10.365342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1367b8-13c5-4440-8ad5-2c15889fb578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:10.366396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1367b8-13c5-4440-8ad5-2c15889fb578 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:10.366924 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1367b8-13c5-4440-8ad5-2c15889fb578 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:10.367407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1367b8-13c5-4440-8ad5-2c15889fb578 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:10.368365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1367b8-13c5-4440-8ad5-2c15889fb578 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:10.368865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1367b8-13c5-4440-8ad5-2c15889fb578 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:10.370517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1367b8-13c5-4440-8ad5-2c15889fb578 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:10.389456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 418/845] 10.222.0.46 () {56 vars in 1135 bytes} [Fri Sep 9 12:38:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:10.396403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926b4c7-68e7-4401-b2f4-681c9c09ff7a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:10.499364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926b4c7-68e7-4401-b2f4-681c9c09ff7a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:10.533752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15d75f-a277-4b2b-95a7-43978ae2ac26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:10.534349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15d75f-a277-4b2b-95a7-43978ae2ac26 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:10.534664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15d75f-a277-4b2b-95a7-43978ae2ac26 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:10.534929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15d75f-a277-4b2b-95a7-43978ae2ac26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:10.572152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:10.608758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15d75f-a277-4b2b-95a7-43978ae2ac26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:10.610111 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15d75f-a277-4b2b-95a7-43978ae2ac26 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:10.723777 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 428/846] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:38:10 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:38:10.729348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1367b8-13c5-4440-8ad5-2c15889fb578 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:10.786396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:10.842395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1367b8-13c5-4440-8ad5-2c15889fb578 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:10.877308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7be1ad-4dd1-47f5-abed-50b1b4a1cdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:10.877860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7be1ad-4dd1-47f5-abed-50b1b4a1cdbe None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:10.878136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7be1ad-4dd1-47f5-abed-50b1b4a1cdbe None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:10.878405 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7be1ad-4dd1-47f5-abed-50b1b4a1cdbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:10.888806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7be1ad-4dd1-47f5-abed-50b1b4a1cdbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:10.890234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7be1ad-4dd1-47f5-abed-50b1b4a1cdbe None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-6c7be1ad-4dd1-47f5-abed-50b1b4a1cdbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:10.905627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:10.906612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:10.907497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:10.907497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:10.907497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:10.910633 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 419/847] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:10.912735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15d75f-a277-4b2b-95a7-43978ae2ac26 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:10.969393 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:11.014608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15d75f-a277-4b2b-95a7-43978ae2ac26 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:11.053100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17765a-a795-4214-978a-92fbca136f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:11.054345 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17765a-a795-4214-978a-92fbca136f7d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:11.054658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17765a-a795-4214-978a-92fbca136f7d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:11.055003 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17765a-a795-4214-978a-92fbca136f7d None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:11.055555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17765a-a795-4214-978a-92fbca136f7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:11.055854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17765a-a795-4214-978a-92fbca136f7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:11.057188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17765a-a795-4214-978a-92fbca136f7d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:11.069166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 429/848] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:11.075132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7be1ad-4dd1-47f5-abed-50b1b4a1cdbe None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:11.184789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7be1ad-4dd1-47f5-abed-50b1b4a1cdbe None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:11.186441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:11.220598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47187ad8-5067-47b6-ab69-8ed518affdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:11.221227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47187ad8-5067-47b6-ab69-8ed518affdfa None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:11.221508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47187ad8-5067-47b6-ab69-8ed518affdfa None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:11.221796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47187ad8-5067-47b6-ab69-8ed518affdfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:11.229202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47187ad8-5067-47b6-ab69-8ed518affdfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:11.230718 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47187ad8-5067-47b6-ab69-8ed518affdfa None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-47187ad8-5067-47b6-ab69-8ed518affdfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:11.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:11.245164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:11.247308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:11.247308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:11.247308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:11.247308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:11.247308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:11.247308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:11.247308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:11.247308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:11.250528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 420/849] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:11.250740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17765a-a795-4214-978a-92fbca136f7d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:11.352815 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17765a-a795-4214-978a-92fbca136f7d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:11.368909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:11.386834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d851d0-0fa6-40c8-9f91-916aa500a110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:11.387421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d851d0-0fa6-40c8-9f91-916aa500a110 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:11.387696 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d851d0-0fa6-40c8-9f91-916aa500a110 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:11.387951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d851d0-0fa6-40c8-9f91-916aa500a110 None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:11.388641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d851d0-0fa6-40c8-9f91-916aa500a110 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:11.388943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d851d0-0fa6-40c8-9f91-916aa500a110 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:11.390351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d851d0-0fa6-40c8-9f91-916aa500a110 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:11.403025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 430/850] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:38:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:11.408782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47187ad8-5067-47b6-ab69-8ed518affdfa None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:11.517375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47187ad8-5067-47b6-ab69-8ed518affdfa None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:11.552874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db20e7-db71-4ee6-b9d5-a87f73bf22d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:11.553477 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db20e7-db71-4ee6-b9d5-a87f73bf22d3 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:11.553944 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db20e7-db71-4ee6-b9d5-a87f73bf22d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:11.554310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db20e7-db71-4ee6-b9d5-a87f73bf22d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:11.555066 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db20e7-db71-4ee6-b9d5-a87f73bf22d3 None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=e8e4805f0b1e484c98fb399313f88ef2) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:11.555371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db20e7-db71-4ee6-b9d5-a87f73bf22d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:11.556830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db20e7-db71-4ee6-b9d5-a87f73bf22d3 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:11.570664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 421/851] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:38:11 2022] GET /identity/v3/role_assignments?user.id=e8e4805f0b1e484c98fb399313f88ef2&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:11.690498 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12e96a-8b41-4a93-8883-9d169d1ec12e None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:11.690810 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12e96a-8b41-4a93-8883-9d169d1ec12e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:11.691057 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12e96a-8b41-4a93-8883-9d169d1ec12e None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:11.692691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 431/852] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:11.698432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ced0e8-6e35-4919-a839-e423a12279d8 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:11.698733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ced0e8-6e35-4919-a839-e423a12279d8 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:11.699017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ced0e8-6e35-4919-a839-e423a12279d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:11.719334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-38ced0e8-6e35-4919-a839-e423a12279d8 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:11.720545 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-38ced0e8-6e35-4919-a839-e423a12279d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:11.748058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:11.795977 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 422/853] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:12.069702 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12e96a-8b41-4a93-8883-9d169d1ec12e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:12.145620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560d52d-501d-446f-8239-36c3dbf2f417 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:12.145919 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560d52d-501d-446f-8239-36c3dbf2f417 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:12.146182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560d52d-501d-446f-8239-36c3dbf2f417 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:12.147913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 423/854] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:12.153047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77398f7-b56f-438e-826e-b4a823b77bbd None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:12.153340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77398f7-b56f-438e-826e-b4a823b77bbd None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:12.153624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77398f7-b56f-438e-826e-b4a823b77bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:12.172985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-e77398f7-b56f-438e-826e-b4a823b77bbd None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:12.174185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e77398f7-b56f-438e-826e-b4a823b77bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:12.187273 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12e96a-8b41-4a93-8883-9d169d1ec12e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:12.221726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1627a8-cb45-4c8b-a9c0-751deff37b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:12.222358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1627a8-cb45-4c8b-a9c0-751deff37b4d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:12.222684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1627a8-cb45-4c8b-a9c0-751deff37b4d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:12.222965 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1627a8-cb45-4c8b-a9c0-751deff37b4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:12.233876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1627a8-cb45-4c8b-a9c0-751deff37b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-6e1627a8-cb45-4c8b-a9c0-751deff37b4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:12.238807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:12.239765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:12.240619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:12.240619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:12.240619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:12.240619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:12.240619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:12.240619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:12.240619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:12.240619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:12.240619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:12.240619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:12.240619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:38:12.240619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:12.240619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 432/855] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:38:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:12.247414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1627a8-cb45-4c8b-a9c0-751deff37b4d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:12.258320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 424/856] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:12.289456 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:12.363089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1627a8-cb45-4c8b-a9c0-751deff37b4d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:12.398828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b554e96-5b5b-442c-a719-5661c44b5153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:12.399431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b554e96-5b5b-442c-a719-5661c44b5153 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:12.399710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b554e96-5b5b-442c-a719-5661c44b5153 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:12.399981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b554e96-5b5b-442c-a719-5661c44b5153 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:12.400734 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b554e96-5b5b-442c-a719-5661c44b5153 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:12.401035 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b554e96-5b5b-442c-a719-5661c44b5153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:12.402460 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b554e96-5b5b-442c-a719-5661c44b5153 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:12.411015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 433/857] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:38:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:12.416977 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77398f7-b56f-438e-826e-b4a823b77bbd None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:12.454441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b5b11-15ed-4559-abdd-ca19ec96400b None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:12.454780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b5b11-15ed-4559-abdd-ca19ec96400b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:12.455098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b5b11-15ed-4559-abdd-ca19ec96400b None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:12.456742 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 434/858] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:38:12.462320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4c914-ada5-4f5a-a677-1d0687685e6e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:12.462627 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4c914-ada5-4f5a-a677-1d0687685e6e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:12.462958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4c914-ada5-4f5a-a677-1d0687685e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:12.482629 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-c8a4c914-ada5-4f5a-a677-1d0687685e6e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:12.483997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c8a4c914-ada5-4f5a-a677-1d0687685e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:12.511028 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:12.534514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77398f7-b56f-438e-826e-b4a823b77bbd None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:12.564677 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c155a72-da1a-4a8d-aa77-bb45a3f779a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:12.564927 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 435/859] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:12.565327 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155a72-da1a-4a8d-aa77-bb45a3f779a8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:12.565610 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155a72-da1a-4a8d-aa77-bb45a3f779a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:12.565791 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4c914-ada5-4f5a-a677-1d0687685e6e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:12.565962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155a72-da1a-4a8d-aa77-bb45a3f779a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:12.580634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c155a72-da1a-4a8d-aa77-bb45a3f779a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:12.582139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c155a72-da1a-4a8d-aa77-bb45a3f779a8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:12.597582 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-1c155a72-da1a-4a8d-aa77-bb45a3f779a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:12.599619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:12.601319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:12.602909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:12.602909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:12.602909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:12.602909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 425/860] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:12.610445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c155a72-da1a-4a8d-aa77-bb45a3f779a8 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:12.674571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4c914-ada5-4f5a-a677-1d0687685e6e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:12.730443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24df5d-6c55-4ed9-914b-31ea0adc3330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:12.731495 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24df5d-6c55-4ed9-914b-31ea0adc3330 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:12.732037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24df5d-6c55-4ed9-914b-31ea0adc3330 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:12.732310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24df5d-6c55-4ed9-914b-31ea0adc3330 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:12.732930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24df5d-6c55-4ed9-914b-31ea0adc3330 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:12.735206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24df5d-6c55-4ed9-914b-31ea0adc3330 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:12.736990 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c155a72-da1a-4a8d-aa77-bb45a3f779a8 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-1c24df5d-6c55-4ed9-914b-31ea0adc3330 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:12.747185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:12.748126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:38:12.749028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:12.754505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 436/861] 10.222.0.46 () {56 vars in 1132 bytes} [Fri Sep 9 12:38:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:12.754505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24df5d-6c55-4ed9-914b-31ea0adc3330 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:12.779628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffed3f8-5bb4-4a24-8e96-14d54262a53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:12.780261 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffed3f8-5bb4-4a24-8e96-14d54262a53e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:12.780517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffed3f8-5bb4-4a24-8e96-14d54262a53e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:12.780790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffed3f8-5bb4-4a24-8e96-14d54262a53e None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:12.781416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffed3f8-5bb4-4a24-8e96-14d54262a53e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:12.781713 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffed3f8-5bb4-4a24-8e96-14d54262a53e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:12.783103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffed3f8-5bb4-4a24-8e96-14d54262a53e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:12.791512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:12.796809 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 426/862] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:12.803943 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffed3f8-5bb4-4a24-8e96-14d54262a53e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:12.862137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:12.872426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24df5d-6c55-4ed9-914b-31ea0adc3330 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:12.909694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7712812-bbca-4905-a1e0-0f8236403a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:12.910304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7712812-bbca-4905-a1e0-0f8236403a3e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:12.910592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7712812-bbca-4905-a1e0-0f8236403a3e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:12.910957 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7712812-bbca-4905-a1e0-0f8236403a3e None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:12.911485 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7712812-bbca-4905-a1e0-0f8236403a3e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:12.911783 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7712812-bbca-4905-a1e0-0f8236403a3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:12.913486 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7712812-bbca-4905-a1e0-0f8236403a3e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:12.916281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffed3f8-5bb4-4a24-8e96-14d54262a53e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:12.918450 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 437/863] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 12:38:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:12.919387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7712812-bbca-4905-a1e0-0f8236403a3e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:12.959810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e264576-f6b5-44c7-a401-47251edc87c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:12.961183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e264576-f6b5-44c7-a401-47251edc87c6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:12.961459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e264576-f6b5-44c7-a401-47251edc87c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:12.961724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e264576-f6b5-44c7-a401-47251edc87c6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:12.969342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e264576-f6b5-44c7-a401-47251edc87c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:12.970759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e264576-f6b5-44c7-a401-47251edc87c6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-2e264576-f6b5-44c7-a401-47251edc87c6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:12.982201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:12.983501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:12.984456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:12.984456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:12.984456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:12.984456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:12.984456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:12.984456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:12.984456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:12.984456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:12.984456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:38:12.984456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:12.984456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 427/864] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 12:38:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:12.985059 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e264576-f6b5-44c7-a401-47251edc87c6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:13.030073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7712812-bbca-4905-a1e0-0f8236403a3e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:13.046416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:13.065492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875afbb8-cd78-4c7b-80a1-53127fb09b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:13.066093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875afbb8-cd78-4c7b-80a1-53127fb09b80 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:13.066382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875afbb8-cd78-4c7b-80a1-53127fb09b80 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:13.066670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875afbb8-cd78-4c7b-80a1-53127fb09b80 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:13.067311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875afbb8-cd78-4c7b-80a1-53127fb09b80 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:13.068641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875afbb8-cd78-4c7b-80a1-53127fb09b80 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:13.094900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e264576-f6b5-44c7-a401-47251edc87c6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:13.132162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d02da-bd6c-45cd-8ed0-5ab05881e344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:13.132769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d02da-bd6c-45cd-8ed0-5ab05881e344 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:13.133048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d02da-bd6c-45cd-8ed0-5ab05881e344 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:13.133316 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d02da-bd6c-45cd-8ed0-5ab05881e344 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:13.134059 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d02da-bd6c-45cd-8ed0-5ab05881e344 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:13.134326 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d02da-bd6c-45cd-8ed0-5ab05881e344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:13.135615 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d02da-bd6c-45cd-8ed0-5ab05881e344 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:13.142530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 438/865] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:38:12 2022] POST /identity/v3/services => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:38:13.142530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875afbb8-cd78-4c7b-80a1-53127fb09b80 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:13.146157 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 428/866] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 12:38:12 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:13.182774 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:13.296823 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875afbb8-cd78-4c7b-80a1-53127fb09b80 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:13.340738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffe427-1aec-4ef1-8004-05f44dacd4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:13.341361 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffe427-1aec-4ef1-8004-05f44dacd4a4 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:13.341636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffe427-1aec-4ef1-8004-05f44dacd4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:13.341906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffe427-1aec-4ef1-8004-05f44dacd4a4 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:13.342749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffe427-1aec-4ef1-8004-05f44dacd4a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:13.343063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffe427-1aec-4ef1-8004-05f44dacd4a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:13.344608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffe427-1aec-4ef1-8004-05f44dacd4a4 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:13.364153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 439/867] 10.222.0.46 () {56 vars in 1149 bytes} [Fri Sep 9 12:38:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:13.371272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d02da-bd6c-45cd-8ed0-5ab05881e344 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:13.481236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d02da-bd6c-45cd-8ed0-5ab05881e344 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:13.508883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:13.521282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13beff-07a8-4fee-9837-45c5ea59b6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:13.521892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13beff-07a8-4fee-9837-45c5ea59b6a4 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:13.522175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13beff-07a8-4fee-9837-45c5ea59b6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:13.522557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13beff-07a8-4fee-9837-45c5ea59b6a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:13.593783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13beff-07a8-4fee-9837-45c5ea59b6a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:13.595591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13beff-07a8-4fee-9837-45c5ea59b6a4 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:13.699771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 429/868] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:38:13 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:38:13.705268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffe427-1aec-4ef1-8004-05f44dacd4a4 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:13.779960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:13.819354 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffe427-1aec-4ef1-8004-05f44dacd4a4 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:13.855338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ac3f0-b3e0-4bf7-a4e3-6e51087e918d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:13.855892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ac3f0-b3e0-4bf7-a4e3-6e51087e918d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:13.856175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ac3f0-b3e0-4bf7-a4e3-6e51087e918d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:13.856444 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ac3f0-b3e0-4bf7-a4e3-6e51087e918d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:13.866808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ac3f0-b3e0-4bf7-a4e3-6e51087e918d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:13.868218 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ac3f0-b3e0-4bf7-a4e3-6e51087e918d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-7a8ac3f0-b3e0-4bf7-a4e3-6e51087e918d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:13.882866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:13.883833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:13.884729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:13.884729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:13.884729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:13.884729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 440/869] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:13.889702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13beff-07a8-4fee-9837-45c5ea59b6a4 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:13.916469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:13.917539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ad937-b13a-45a2-ae03-25d91dfbec3d None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:13.917826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ad937-b13a-45a2-ae03-25d91dfbec3d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:13.918103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ad937-b13a-45a2-ae03-25d91dfbec3d None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:13.920327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 441/870] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:38:13.928380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7192d38-6f20-43b3-8dbe-a95472ca7f04 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:13.928380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7192d38-6f20-43b3-8dbe-a95472ca7f04 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:13.928380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7192d38-6f20-43b3-8dbe-a95472ca7f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:13.947874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-f7192d38-6f20-43b3-8dbe-a95472ca7f04 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:13.949074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f7192d38-6f20-43b3-8dbe-a95472ca7f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:13.998937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13beff-07a8-4fee-9837-45c5ea59b6a4 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:14.036357 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986ce7e-ead5-495c-ab00-8964be5dbb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:14.036966 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986ce7e-ead5-495c-ab00-8964be5dbb9b None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:14.037292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986ce7e-ead5-495c-ab00-8964be5dbb9b None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:14.037617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986ce7e-ead5-495c-ab00-8964be5dbb9b None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:14.038315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986ce7e-ead5-495c-ab00-8964be5dbb9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:14.038651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986ce7e-ead5-495c-ab00-8964be5dbb9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:14.040143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986ce7e-ead5-495c-ab00-8964be5dbb9b None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:14.042573 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 442/871] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:13 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:14.056650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 430/872] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:14.063567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7192d38-6f20-43b3-8dbe-a95472ca7f04 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:14.134282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:14.185850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7192d38-6f20-43b3-8dbe-a95472ca7f04 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:14.222076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcac15-8f2c-4de3-87c0-fb90091cd58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:14.223075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcac15-8f2c-4de3-87c0-fb90091cd58c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:14.223587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcac15-8f2c-4de3-87c0-fb90091cd58c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:14.224086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcac15-8f2c-4de3-87c0-fb90091cd58c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:14.231705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcac15-8f2c-4de3-87c0-fb90091cd58c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:14.233396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcac15-8f2c-4de3-87c0-fb90091cd58c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-aebcac15-8f2c-4de3-87c0-fb90091cd58c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:14.245659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:14.246608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:14.247513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:14.247513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:14.247513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:14.247513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:14.247513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:14.247513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:14.247513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:14.247513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:14.249784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 443/873] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:14.255705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986ce7e-ead5-495c-ab00-8964be5dbb9b None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:14.344725 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:14.360207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcac15-8f2c-4de3-87c0-fb90091cd58c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:14.363721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986ce7e-ead5-495c-ab00-8964be5dbb9b None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:14.406384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ccb59-e40f-4bf3-a3ac-8d5ea872b961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:14.406945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ccb59-e40f-4bf3-a3ac-8d5ea872b961 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:14.407224 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ccb59-e40f-4bf3-a3ac-8d5ea872b961 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:14.407496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ccb59-e40f-4bf3-a3ac-8d5ea872b961 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:14.408541 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ccb59-e40f-4bf3-a3ac-8d5ea872b961 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:14.408853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ccb59-e40f-4bf3-a3ac-8d5ea872b961 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:14.410611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ccb59-e40f-4bf3-a3ac-8d5ea872b961 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:14.430520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 431/874] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:38:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:14.435172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ccb59-e40f-4bf3-a3ac-8d5ea872b961 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:14.489242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcac15-8f2c-4de3-87c0-fb90091cd58c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:14.504063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:14.526670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1969bf-0913-4980-8920-943ce0a8a68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:14.527364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1969bf-0913-4980-8920-943ce0a8a68e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:14.527651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1969bf-0913-4980-8920-943ce0a8a68e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:14.527923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1969bf-0913-4980-8920-943ce0a8a68e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:14.528510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1969bf-0913-4980-8920-943ce0a8a68e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:14.530230 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1969bf-0913-4980-8920-943ce0a8a68e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-5b1969bf-0913-4980-8920-943ce0a8a68e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:14.542385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:14.543664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:14.544720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:14.551398 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 444/875] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 12:38:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:14.554372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1969bf-0913-4980-8920-943ce0a8a68e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:14.567539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ccb59-e40f-4bf3-a3ac-8d5ea872b961 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:14.612964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c2a51-5d09-4bc9-991e-8d85ee56cb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:14.614711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c2a51-5d09-4bc9-991e-8d85ee56cb9f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:14.615204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c2a51-5d09-4bc9-991e-8d85ee56cb9f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:14.615703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c2a51-5d09-4bc9-991e-8d85ee56cb9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:14.616753 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c2a51-5d09-4bc9-991e-8d85ee56cb9f None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=ec77f4e9939b4dd685b850f8b6fb1e56) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:14.617269 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c2a51-5d09-4bc9-991e-8d85ee56cb9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:14.619770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c2a51-5d09-4bc9-991e-8d85ee56cb9f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:14.634724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 432/876] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:38:14 2022] GET /identity/v3/role_assignments?user.id=ec77f4e9939b4dd685b850f8b6fb1e56&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:14.673950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1969bf-0913-4980-8920-943ce0a8a68e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:14.712327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe770f21-a269-4977-b5cd-23be81190a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:14.713023 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe770f21-a269-4977-b5cd-23be81190a47 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:14.713299 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe770f21-a269-4977-b5cd-23be81190a47 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:14.713567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe770f21-a269-4977-b5cd-23be81190a47 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:14.714276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe770f21-a269-4977-b5cd-23be81190a47 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:14.714606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe770f21-a269-4977-b5cd-23be81190a47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:14.716049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe770f21-a269-4977-b5cd-23be81190a47 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:14.721694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 445/877] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 12:38:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:14.728302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c2a51-5d09-4bc9-991e-8d85ee56cb9f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:14.740004 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:14.846457 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c2a51-5d09-4bc9-991e-8d85ee56cb9f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:14.883241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e622e-0382-43e0-8fef-1362cb96f5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:14.883857 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e622e-0382-43e0-8fef-1362cb96f5b0 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:14.884141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e622e-0382-43e0-8fef-1362cb96f5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:14.884469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e622e-0382-43e0-8fef-1362cb96f5b0 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:14.885047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e622e-0382-43e0-8fef-1362cb96f5b0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:14.885350 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e622e-0382-43e0-8fef-1362cb96f5b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:14.886757 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e622e-0382-43e0-8fef-1362cb96f5b0 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:14.891886 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 433/878] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 12:38:14 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:15.422320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4f6be-4e53-4a19-8cd8-cf749b685642 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:15.422829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4f6be-4e53-4a19-8cd8-cf749b685642 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:15.422829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4f6be-4e53-4a19-8cd8-cf749b685642 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:15.424409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 446/879] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:15.429934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fa32b-3d56-41b6-9816-14ecffa98112 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:15.430227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fa32b-3d56-41b6-9816-14ecffa98112 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:15.430518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fa32b-3d56-41b6-9816-14ecffa98112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:15.449018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-670fa32b-3d56-41b6-9816-14ecffa98112 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:15.450194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-670fa32b-3d56-41b6-9816-14ecffa98112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:15.502994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa62634-3248-45d2-882a-646b21f19f5b None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:15.503267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa62634-3248-45d2-882a-646b21f19f5b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:15.503530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa62634-3248-45d2-882a-646b21f19f5b None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:15.505090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 447/880] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:15.510587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cced84-58e3-4a10-b37a-592c2d69157c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:15.510930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cced84-58e3-4a10-b37a-592c2d69157c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:15.511208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cced84-58e3-4a10-b37a-592c2d69157c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:15.516915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:15.522153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 434/881] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:15.530891 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-f3cced84-58e3-4a10-b37a-592c2d69157c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:15.532068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f3cced84-58e3-4a10-b37a-592c2d69157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:15.572025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:15.605644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 448/882] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:15.797078 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fa32b-3d56-41b6-9816-14ecffa98112 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:15.891465 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cced84-58e3-4a10-b37a-592c2d69157c None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:15.927272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fa32b-3d56-41b6-9816-14ecffa98112 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:15.977043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e2f74-5118-457c-89ee-5760a1ffb671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:15.977766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e2f74-5118-457c-89ee-5760a1ffb671 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:15.978053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e2f74-5118-457c-89ee-5760a1ffb671 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:15.978329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e2f74-5118-457c-89ee-5760a1ffb671 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:15.979143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e2f74-5118-457c-89ee-5760a1ffb671 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:15.980602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e2f74-5118-457c-89ee-5760a1ffb671 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:16.017814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cced84-58e3-4a10-b37a-592c2d69157c None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:16.057026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10448799-e1b5-44c9-83ce-7679a0b78cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:16.057582 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10448799-e1b5-44c9-83ce-7679a0b78cc8 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:16.057859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10448799-e1b5-44c9-83ce-7679a0b78cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:16.058125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10448799-e1b5-44c9-83ce-7679a0b78cc8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:16.058747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10448799-e1b5-44c9-83ce-7679a0b78cc8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:16.060138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10448799-e1b5-44c9-83ce-7679a0b78cc8 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-10448799-e1b5-44c9-83ce-7679a0b78cc8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:16.070703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:16.071814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:38:16.073470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:16.074267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 435/883] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:38:15 2022] POST /identity/v3/services => generated 269 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:38:16.074344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 449/884] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:38:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:16.078138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10448799-e1b5-44c9-83ce-7679a0b78cc8 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:16.130533 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:16.211167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10448799-e1b5-44c9-83ce-7679a0b78cc8 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:16.265101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623b679-a44d-4799-809a-abfbcf84821b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:16.265718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623b679-a44d-4799-809a-abfbcf84821b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:16.266016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623b679-a44d-4799-809a-abfbcf84821b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:16.266286 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623b679-a44d-4799-809a-abfbcf84821b None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:16.266937 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623b679-a44d-4799-809a-abfbcf84821b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:16.267233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623b679-a44d-4799-809a-abfbcf84821b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:16.268566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623b679-a44d-4799-809a-abfbcf84821b None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:16.274377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 450/885] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:38:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:16.280257 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e2f74-5118-457c-89ee-5760a1ffb671 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:16.348896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:16.394921 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e2f74-5118-457c-89ee-5760a1ffb671 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:16.433560 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502901a-76bd-4fff-807a-817f1561ab67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:16.434629 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502901a-76bd-4fff-807a-817f1561ab67 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:16.435211 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502901a-76bd-4fff-807a-817f1561ab67 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:16.435948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502901a-76bd-4fff-807a-817f1561ab67 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:16.436753 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502901a-76bd-4fff-807a-817f1561ab67 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:16.437226 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502901a-76bd-4fff-807a-817f1561ab67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:16.438769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502901a-76bd-4fff-807a-817f1561ab67 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:16.446676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 436/886] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:38:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:16.450752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623b679-a44d-4799-809a-abfbcf84821b None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:16.549198 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:16.557979 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623b679-a44d-4799-809a-abfbcf84821b None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:16.597421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fa787-b0be-4666-ba27-49e0c643bb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:16.598029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fa787-b0be-4666-ba27-49e0c643bb7f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:16.598301 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fa787-b0be-4666-ba27-49e0c643bb7f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:16.598592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fa787-b0be-4666-ba27-49e0c643bb7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:16.599343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fa787-b0be-4666-ba27-49e0c643bb7f None admin] RBAC: Adding query filter params (interface=public, service_id=c2a0badd94844ebcb5b0168c2aef1e1a, region_id=RegionOne) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:16.599589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fa787-b0be-4666-ba27-49e0c643bb7f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:16.601461 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fa787-b0be-4666-ba27-49e0c643bb7f None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:16.607246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 451/887] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 12:38:16 2022] GET /identity/v3/endpoints?service_id=c2a0badd94844ebcb5b0168c2aef1e1a&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:16.615788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502901a-76bd-4fff-807a-817f1561ab67 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:16.724531 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502901a-76bd-4fff-807a-817f1561ab67 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:16.754454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:16.766422 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f24d0-f439-4385-a60a-77667949b33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:16.771557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f24d0-f439-4385-a60a-77667949b33a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:16.771557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f24d0-f439-4385-a60a-77667949b33a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:16.771557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f24d0-f439-4385-a60a-77667949b33a None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:16.771557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f24d0-f439-4385-a60a-77667949b33a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:16.771557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f24d0-f439-4385-a60a-77667949b33a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:16.780828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 437/888] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 12:38:16 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:38:17.076833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69a63f-d099-48fd-a535-7e6c49de6046 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:17.077259 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69a63f-d099-48fd-a535-7e6c49de6046 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:17.077364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69a63f-d099-48fd-a535-7e6c49de6046 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:17.079107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 452/889] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:17.085877 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013db537-b5d6-499d-b68b-be263eca3311 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:17.086898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013db537-b5d6-499d-b68b-be263eca3311 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:17.087473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013db537-b5d6-499d-b68b-be263eca3311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:17.107489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-013db537-b5d6-499d-b68b-be263eca3311 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:17.109066 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-013db537-b5d6-499d-b68b-be263eca3311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:17.181979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 438/890] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:17.191295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d110032-3f6b-4904-b61a-41515e60dc1a None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:17.191846 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d110032-3f6b-4904-b61a-41515e60dc1a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:17.192334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d110032-3f6b-4904-b61a-41515e60dc1a None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:17.194626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 453/891] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:17.199943 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd869287-b587-42d6-aae7-f3ad6a679622 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:17.200499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd869287-b587-42d6-aae7-f3ad6a679622 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:17.200990 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd869287-b587-42d6-aae7-f3ad6a679622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:17.219941 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-cd869287-b587-42d6-aae7-f3ad6a679622 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:17.221437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-cd869287-b587-42d6-aae7-f3ad6a679622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:17.298430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 439/892] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:17.456391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d110032-3f6b-4904-b61a-41515e60dc1a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:17.560239 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d110032-3f6b-4904-b61a-41515e60dc1a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:17.576049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd869287-b587-42d6-aae7-f3ad6a679622 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:17.594828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5e7f4-d2bf-4631-b60c-860ed261fdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:17.595454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5e7f4-d2bf-4631-b60c-860ed261fdbf None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:17.595791 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5e7f4-d2bf-4631-b60c-860ed261fdbf None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:17.595980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5e7f4-d2bf-4631-b60c-860ed261fdbf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:17.606738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5e7f4-d2bf-4631-b60c-860ed261fdbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-e7d5e7f4-d2bf-4631-b60c-860ed261fdbf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:17.611412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:17.612472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:17.613378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:17.613378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:17.613378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:17.613378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:17.613378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:17.613378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:17.613378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:17.613378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:17.613378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:17.613378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:17.613378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:38:17.613378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:17.617916 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 454/893] 10.222.0.46 () {56 vars in 1120 bytes} [Fri Sep 9 12:38:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:17.619243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5e7f4-d2bf-4631-b60c-860ed261fdbf None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:17.634599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:17.646887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:17.690191 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd869287-b587-42d6-aae7-f3ad6a679622 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:17.723811 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5e7f4-d2bf-4631-b60c-860ed261fdbf None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:17.729307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce2037-1096-4773-b46a-2d9d9329c462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:17.730075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce2037-1096-4773-b46a-2d9d9329c462 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:17.730518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce2037-1096-4773-b46a-2d9d9329c462 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:17.730892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce2037-1096-4773-b46a-2d9d9329c462 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:17.731652 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce2037-1096-4773-b46a-2d9d9329c462 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:17.733173 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce2037-1096-4773-b46a-2d9d9329c462 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:17.762166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98209cfe-165a-4d38-8746-c7cac1fe8199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:17.763215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98209cfe-165a-4d38-8746-c7cac1fe8199 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:17.763651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98209cfe-165a-4d38-8746-c7cac1fe8199 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:17.764063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98209cfe-165a-4d38-8746-c7cac1fe8199 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:17.765011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98209cfe-165a-4d38-8746-c7cac1fe8199 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:17.765463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98209cfe-165a-4d38-8746-c7cac1fe8199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:17.766987 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98209cfe-165a-4d38-8746-c7cac1fe8199 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:17.780371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 455/894] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 12:38:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:17.781831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98209cfe-165a-4d38-8746-c7cac1fe8199 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:17.797256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 440/895] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:38:17 2022] POST /identity/v3/services => generated 241 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:38:17.884353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:17.906290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98209cfe-165a-4d38-8746-c7cac1fe8199 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:17.956125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681062c3-80d3-4fce-afd8-d21209dfc29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:17.956760 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681062c3-80d3-4fce-afd8-d21209dfc29d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:17.957036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681062c3-80d3-4fce-afd8-d21209dfc29d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:17.957305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681062c3-80d3-4fce-afd8-d21209dfc29d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:17.969321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681062c3-80d3-4fce-afd8-d21209dfc29d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:17.970794 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681062c3-80d3-4fce-afd8-d21209dfc29d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-681062c3-80d3-4fce-afd8-d21209dfc29d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:17.986511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:17.987521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:17.988584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:17.988584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:17.988584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:17.988584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 456/896] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:17.995166 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce2037-1096-4773-b46a-2d9d9329c462 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:18.101591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce2037-1096-4773-b46a-2d9d9329c462 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:18.110885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:18.136682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0026be-4edf-4cda-be48-734e4ad1f1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:18.137435 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0026be-4edf-4cda-be48-734e4ad1f1de None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:18.137827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0026be-4edf-4cda-be48-734e4ad1f1de None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:18.138198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0026be-4edf-4cda-be48-734e4ad1f1de None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:18.138971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0026be-4edf-4cda-be48-734e4ad1f1de None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:18.139374 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0026be-4edf-4cda-be48-734e4ad1f1de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:18.140842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0026be-4edf-4cda-be48-734e4ad1f1de None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:18.153863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 441/897] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:18.159500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681062c3-80d3-4fce-afd8-d21209dfc29d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:18.275720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681062c3-80d3-4fce-afd8-d21209dfc29d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:18.304837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:18.313866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95a013-b301-4208-8374-c00e1175fcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:18.314632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95a013-b301-4208-8374-c00e1175fcbc None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:18.317073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95a013-b301-4208-8374-c00e1175fcbc None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:18.317073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95a013-b301-4208-8374-c00e1175fcbc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:18.324369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95a013-b301-4208-8374-c00e1175fcbc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:18.325767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95a013-b301-4208-8374-c00e1175fcbc None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-5e95a013-b301-4208-8374-c00e1175fcbc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:18.337264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:18.338254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:18.339250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:18.339250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:18.339250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:18.339250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:18.339250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:18.339250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:18.339250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:18.339250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:18.339250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:38:18.339250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:18.339250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 457/898] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 12:38:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:18.344643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0026be-4edf-4cda-be48-734e4ad1f1de None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:18.376165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77493a9c-6f87-40f2-8b50-4bd7105384c9 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:18.376444 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77493a9c-6f87-40f2-8b50-4bd7105384c9 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:18.376717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77493a9c-6f87-40f2-8b50-4bd7105384c9 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:18.378051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 458/899] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:38:18.385259 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba48f2-27d7-499e-ac1d-31385e4d58b9 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:18.385539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba48f2-27d7-499e-ac1d-31385e4d58b9 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:18.385807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba48f2-27d7-499e-ac1d-31385e4d58b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:18.404341 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-47ba48f2-27d7-499e-ac1d-31385e4d58b9 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:18.405524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-47ba48f2-27d7-499e-ac1d-31385e4d58b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:18.465424 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0026be-4edf-4cda-be48-734e4ad1f1de None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:18.479388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 459/900] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:18.508824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81220f2e-f943-462e-9657-6a3718bde5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:18.509607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81220f2e-f943-462e-9657-6a3718bde5dd None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:18.509958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81220f2e-f943-462e-9657-6a3718bde5dd None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:18.510299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81220f2e-f943-462e-9657-6a3718bde5dd None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:18.511119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81220f2e-f943-462e-9657-6a3718bde5dd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:18.511498 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81220f2e-f943-462e-9657-6a3718bde5dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:18.512977 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81220f2e-f943-462e-9657-6a3718bde5dd None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:18.532029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 442/901] 10.222.0.46 () {56 vars in 1149 bytes} [Fri Sep 9 12:38:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:18.539616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba48f2-27d7-499e-ac1d-31385e4d58b9 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:18.647229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba48f2-27d7-499e-ac1d-31385e4d58b9 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:18.665914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:18.684027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe65d5-eb6e-4832-b8e7-ddce6186ccea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:18.691149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe65d5-eb6e-4832-b8e7-ddce6186ccea None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:18.691149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe65d5-eb6e-4832-b8e7-ddce6186ccea None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:18.691149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe65d5-eb6e-4832-b8e7-ddce6186ccea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:18.755286 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe65d5-eb6e-4832-b8e7-ddce6186ccea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:18.756738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe65d5-eb6e-4832-b8e7-ddce6186ccea None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:18.763704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81220f2e-f943-462e-9657-6a3718bde5dd None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:18.805721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:18.867387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 460/902] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:38:18 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:38:18.868652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe65d5-eb6e-4832-b8e7-ddce6186ccea None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:18.886385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:18.892348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81220f2e-f943-462e-9657-6a3718bde5dd None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:18.931611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f0123-9e0c-4a50-b40f-7a795f5dcd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:18.932542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f0123-9e0c-4a50-b40f-7a795f5dcd4c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:18.933046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f0123-9e0c-4a50-b40f-7a795f5dcd4c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:18.933510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f0123-9e0c-4a50-b40f-7a795f5dcd4c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:18.934269 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f0123-9e0c-4a50-b40f-7a795f5dcd4c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:18.936223 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f0123-9e0c-4a50-b40f-7a795f5dcd4c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-c35f0123-9e0c-4a50-b40f-7a795f5dcd4c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:18.946225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:18.947291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:38:18.948221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:18.951192 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 443/903] 10.222.0.46 () {56 vars in 1132 bytes} [Fri Sep 9 12:38:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:18.958546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f0123-9e0c-4a50-b40f-7a795f5dcd4c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:18.972127 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe65d5-eb6e-4832-b8e7-ddce6186ccea None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:19.008803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164d41c-73f2-4e12-b1f7-e8c4c1268dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:19.009432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164d41c-73f2-4e12-b1f7-e8c4c1268dd0 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:19.009732 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164d41c-73f2-4e12-b1f7-e8c4c1268dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:19.010051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164d41c-73f2-4e12-b1f7-e8c4c1268dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:19.020326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164d41c-73f2-4e12-b1f7-e8c4c1268dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:19.021743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164d41c-73f2-4e12-b1f7-e8c4c1268dd0 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-2164d41c-73f2-4e12-b1f7-e8c4c1268dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:19.037660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:19.039095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:19.040217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:19.040217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:19.040217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:19.042616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 461/904] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:19.044955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164d41c-73f2-4e12-b1f7-e8c4c1268dd0 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:19.077530 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:19.089450 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f0123-9e0c-4a50-b40f-7a795f5dcd4c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:19.133155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:19.136994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da92004-9343-4dd2-8a30-6b7f8efd6c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:19.137611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da92004-9343-4dd2-8a30-6b7f8efd6c4e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:19.137890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da92004-9343-4dd2-8a30-6b7f8efd6c4e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:19.138163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da92004-9343-4dd2-8a30-6b7f8efd6c4e None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:19.138823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da92004-9343-4dd2-8a30-6b7f8efd6c4e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:19.139380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da92004-9343-4dd2-8a30-6b7f8efd6c4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:19.140487 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da92004-9343-4dd2-8a30-6b7f8efd6c4e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:19.150451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 444/905] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 12:38:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:19.150451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec0401-1d16-44ce-a01e-bcf13ed53791 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:19.150451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec0401-1d16-44ce-a01e-bcf13ed53791 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:19.150451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec0401-1d16-44ce-a01e-bcf13ed53791 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:19.150451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 445/906] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:19.151215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec0401-1d16-44ce-a01e-bcf13ed53791 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:19.174519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164d41c-73f2-4e12-b1f7-e8c4c1268dd0 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:19.206864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f445f-d569-4ef2-9ec8-17df027c5112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:19.207565 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f445f-d569-4ef2-9ec8-17df027c5112 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:19.207843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f445f-d569-4ef2-9ec8-17df027c5112 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:19.208110 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f445f-d569-4ef2-9ec8-17df027c5112 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:19.208751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f445f-d569-4ef2-9ec8-17df027c5112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:19.209043 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f445f-d569-4ef2-9ec8-17df027c5112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:19.211152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f445f-d569-4ef2-9ec8-17df027c5112 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:19.223423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 462/907] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:19.224740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84d44e-ff14-4bae-b82c-27c88dc82d60 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:19.225020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84d44e-ff14-4bae-b82c-27c88dc82d60 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:19.225338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84d44e-ff14-4bae-b82c-27c88dc82d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:19.243430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-4d84d44e-ff14-4bae-b82c-27c88dc82d60 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:19.244603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4d84d44e-ff14-4bae-b82c-27c88dc82d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:19.283953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec0401-1d16-44ce-a01e-bcf13ed53791 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:19.315122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 463/908] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:19.321551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc57396-ac58-482b-b5f5-bbb3c30068b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:19.322162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc57396-ac58-482b-b5f5-bbb3c30068b1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:19.322444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc57396-ac58-482b-b5f5-bbb3c30068b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:19.322842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc57396-ac58-482b-b5f5-bbb3c30068b1 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:19.323470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84d44e-ff14-4bae-b82c-27c88dc82d60 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:19.323572 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc57396-ac58-482b-b5f5-bbb3c30068b1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:19.323771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc57396-ac58-482b-b5f5-bbb3c30068b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:19.325132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc57396-ac58-482b-b5f5-bbb3c30068b1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:19.336298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 446/909] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 12:38:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:19.337085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc57396-ac58-482b-b5f5-bbb3c30068b1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:19.363423 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:19.399838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:19.447900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc57396-ac58-482b-b5f5-bbb3c30068b1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:19.457852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84d44e-ff14-4bae-b82c-27c88dc82d60 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:19.483200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59ed7c-f6f1-406d-9afb-924b10c9f611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:19.483805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59ed7c-f6f1-406d-9afb-924b10c9f611 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:19.484087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59ed7c-f6f1-406d-9afb-924b10c9f611 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:19.484363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59ed7c-f6f1-406d-9afb-924b10c9f611 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:19.485283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59ed7c-f6f1-406d-9afb-924b10c9f611 None admin] RBAC: Adding query filter params (interface=public, service_id=151ae63b6bab41dea5dc5db47cc8650c, region_id=RegionOne) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:19.485411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59ed7c-f6f1-406d-9afb-924b10c9f611 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:19.486799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59ed7c-f6f1-406d-9afb-924b10c9f611 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:19.498905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 447/910] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 12:38:19 2022] GET /identity/v3/endpoints?service_id=151ae63b6bab41dea5dc5db47cc8650c&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:19.500343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59ed7c-f6f1-406d-9afb-924b10c9f611 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:19.501198 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3e7be-a8f2-4f15-bc8a-4ac629fe5670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:19.502370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3e7be-a8f2-4f15-bc8a-4ac629fe5670 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:19.503047 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3e7be-a8f2-4f15-bc8a-4ac629fe5670 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:19.503517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3e7be-a8f2-4f15-bc8a-4ac629fe5670 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:19.513160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3e7be-a8f2-4f15-bc8a-4ac629fe5670 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:19.515388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3e7be-a8f2-4f15-bc8a-4ac629fe5670 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-64d3e7be-a8f2-4f15-bc8a-4ac629fe5670 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:19.530800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:19.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:19.532719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:19.532719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:19.532719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:19.532719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:19.532719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:19.532719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:19.532719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:19.532719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:19.540334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 464/911] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:19.541934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3e7be-a8f2-4f15-bc8a-4ac629fe5670 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:19.593982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:19.614240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59ed7c-f6f1-406d-9afb-924b10c9f611 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:19.657149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3e7be-a8f2-4f15-bc8a-4ac629fe5670 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:19.659030 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f106c4-21dd-4cf6-b149-169bbf076772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:19.660091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f106c4-21dd-4cf6-b149-169bbf076772 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:19.660091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f106c4-21dd-4cf6-b149-169bbf076772 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:19.660266 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f106c4-21dd-4cf6-b149-169bbf076772 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:19.660764 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f106c4-21dd-4cf6-b149-169bbf076772 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:19.662113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f106c4-21dd-4cf6-b149-169bbf076772 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:19.678520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 448/912] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 12:38:19 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:38:19.678520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f106c4-21dd-4cf6-b149-169bbf076772 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:19.691989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:19.711674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f6a0c-db57-4684-9831-897eed3c225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:19.712312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f6a0c-db57-4684-9831-897eed3c225c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:19.712589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f6a0c-db57-4684-9831-897eed3c225c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:19.712859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f6a0c-db57-4684-9831-897eed3c225c None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:19.713865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f6a0c-db57-4684-9831-897eed3c225c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:19.714165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f6a0c-db57-4684-9831-897eed3c225c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:19.716503 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f6a0c-db57-4684-9831-897eed3c225c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:19.738716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 465/913] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:38:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:19.743551 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f6a0c-db57-4684-9831-897eed3c225c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:19.833893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f106c4-21dd-4cf6-b149-169bbf076772 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:19.870345 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f6a0c-db57-4684-9831-897eed3c225c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:19.878272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f1921-24c9-43b9-9647-001a1db52f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:19.878936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f1921-24c9-43b9-9647-001a1db52f7e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:19.879213 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f1921-24c9-43b9-9647-001a1db52f7e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:19.879478 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f1921-24c9-43b9-9647-001a1db52f7e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:19.880049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f1921-24c9-43b9-9647-001a1db52f7e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:19.881406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f1921-24c9-43b9-9647-001a1db52f7e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-344f1921-24c9-43b9-9647-001a1db52f7e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:19.892958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:19.894210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:19.895260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 449/914] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:38:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:19.900713 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f1921-24c9-43b9-9647-001a1db52f7e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:19.917144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:19.926389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b50ea-58f9-4cd3-a448-5d81a5fe1f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:19.927063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b50ea-58f9-4cd3-a448-5d81a5fe1f55 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:19.931678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b50ea-58f9-4cd3-a448-5d81a5fe1f55 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:19.931678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b50ea-58f9-4cd3-a448-5d81a5fe1f55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:19.931678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b50ea-58f9-4cd3-a448-5d81a5fe1f55 None admin] RBAC: Adding query filter params (role.id=0f979abba91c44569a1ceb70bceceacb, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=ec77f4e9939b4dd685b850f8b6fb1e56) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:19.931678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b50ea-58f9-4cd3-a448-5d81a5fe1f55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:19.931678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b50ea-58f9-4cd3-a448-5d81a5fe1f55 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:19.939466 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 466/915] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:38:19 2022] GET /identity/v3/role_assignments?user.id=ec77f4e9939b4dd685b850f8b6fb1e56&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=0f979abba91c44569a1ceb70bceceacb => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:20.030396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f1921-24c9-43b9-9647-001a1db52f7e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:20.073260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51060a9a-959f-40ff-85f6-689363e91c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:20.073887 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51060a9a-959f-40ff-85f6-689363e91c08 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:20.074163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51060a9a-959f-40ff-85f6-689363e91c08 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:20.074433 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51060a9a-959f-40ff-85f6-689363e91c08 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:20.075096 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51060a9a-959f-40ff-85f6-689363e91c08 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:20.075396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51060a9a-959f-40ff-85f6-689363e91c08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:20.076838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51060a9a-959f-40ff-85f6-689363e91c08 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:20.082858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 450/916] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:38:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:20.089357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b50ea-58f9-4cd3-a448-5d81a5fe1f55 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:20.146547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:20.169360 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060af75-727f-4208-8afb-f5159bcb249b None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:20.169906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060af75-727f-4208-8afb-f5159bcb249b None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:20.169906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060af75-727f-4208-8afb-f5159bcb249b None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:20.171414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 451/917] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:20.177658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aed412-8bd1-4a3f-9859-b6e61f290787 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:20.179167 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aed412-8bd1-4a3f-9859-b6e61f290787 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:20.179595 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aed412-8bd1-4a3f-9859-b6e61f290787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:20.200221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-91aed412-8bd1-4a3f-9859-b6e61f290787 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:20.201588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-91aed412-8bd1-4a3f-9859-b6e61f290787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:20.212602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b50ea-58f9-4cd3-a448-5d81a5fe1f55 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:20.257278 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b289c8-d457-4f3d-a171-e0daccc8e616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:20.258142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b289c8-d457-4f3d-a171-e0daccc8e616 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:20.258911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b289c8-d457-4f3d-a171-e0daccc8e616 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:20.259343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b289c8-d457-4f3d-a171-e0daccc8e616 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:20.260315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b289c8-d457-4f3d-a171-e0daccc8e616 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:20.260770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b289c8-d457-4f3d-a171-e0daccc8e616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:20.262831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b289c8-d457-4f3d-a171-e0daccc8e616 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:20.270038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 467/918] 10.222.0.46 () {56 vars in 1114 bytes} [Fri Sep 9 12:38:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:20.275947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 452/919] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:20.276957 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b289c8-d457-4f3d-a171-e0daccc8e616 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:20.377671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b289c8-d457-4f3d-a171-e0daccc8e616 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:20.391142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:20.412121 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718b6ff-b926-4ae9-8d3b-71ad63d8b2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:20.412739 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718b6ff-b926-4ae9-8d3b-71ad63d8b2e3 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:20.413081 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718b6ff-b926-4ae9-8d3b-71ad63d8b2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:20.413354 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718b6ff-b926-4ae9-8d3b-71ad63d8b2e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:20.414008 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718b6ff-b926-4ae9-8d3b-71ad63d8b2e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:20.415396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718b6ff-b926-4ae9-8d3b-71ad63d8b2e3 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:20.504000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 468/920] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:38:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:38:20.556017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aed412-8bd1-4a3f-9859-b6e61f290787 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:20.559378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:20.672890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aed412-8bd1-4a3f-9859-b6e61f290787 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:20.717223 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cec51-22ce-4251-a7c9-44c9da7ebbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:20.717871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cec51-22ce-4251-a7c9-44c9da7ebbc5 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:20.718154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cec51-22ce-4251-a7c9-44c9da7ebbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:20.718431 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cec51-22ce-4251-a7c9-44c9da7ebbc5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:20.719044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cec51-22ce-4251-a7c9-44c9da7ebbc5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:20.720431 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cec51-22ce-4251-a7c9-44c9da7ebbc5 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-9a6cec51-22ce-4251-a7c9-44c9da7ebbc5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:20.730464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:20.731634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:20.732922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 453/921] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 12:38:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:20.738333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718b6ff-b926-4ae9-8d3b-71ad63d8b2e3 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:20.854005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718b6ff-b926-4ae9-8d3b-71ad63d8b2e3 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:20.891431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9fb92-dc05-4ca9-bae1-2cb9cff74238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:20.892392 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9fb92-dc05-4ca9-bae1-2cb9cff74238 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:20.892885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9fb92-dc05-4ca9-bae1-2cb9cff74238 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:20.893383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9fb92-dc05-4ca9-bae1-2cb9cff74238 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:20.894236 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9fb92-dc05-4ca9-bae1-2cb9cff74238 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:20.894789 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9fb92-dc05-4ca9-bae1-2cb9cff74238 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:20.896329 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9fb92-dc05-4ca9-bae1-2cb9cff74238 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:20.902474 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 469/922] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 12:38:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:20.910420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cec51-22ce-4251-a7c9-44c9da7ebbc5 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:20.973214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:21.012200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cec51-22ce-4251-a7c9-44c9da7ebbc5 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:21.047057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a55797-8664-446e-bf50-e778335ef4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:21.047660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a55797-8664-446e-bf50-e778335ef4c3 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:21.047941 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a55797-8664-446e-bf50-e778335ef4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:21.048214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a55797-8664-446e-bf50-e778335ef4c3 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:21.048837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a55797-8664-446e-bf50-e778335ef4c3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:21.049141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a55797-8664-446e-bf50-e778335ef4c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:21.050519 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a55797-8664-446e-bf50-e778335ef4c3 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:21.055837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 454/923] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 12:38:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:21.061584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9fb92-dc05-4ca9-bae1-2cb9cff74238 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:21.092425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:21.169630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9fb92-dc05-4ca9-bae1-2cb9cff74238 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:21.207783 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d582e-21fe-4a8d-8c67-35183d5c85f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:21.208397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d582e-21fe-4a8d-8c67-35183d5c85f1 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:21.208677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d582e-21fe-4a8d-8c67-35183d5c85f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:21.208949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d582e-21fe-4a8d-8c67-35183d5c85f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:21.210255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d582e-21fe-4a8d-8c67-35183d5c85f1 None admin] RBAC: Adding query filter params (interface=public, service_id=598c6f15c2b14529a392744e6c85617b, region_id=RegionOne) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:21.210600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d582e-21fe-4a8d-8c67-35183d5c85f1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:21.211958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d582e-21fe-4a8d-8c67-35183d5c85f1 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:21.217680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 470/924] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 12:38:21 2022] GET /identity/v3/endpoints?service_id=598c6f15c2b14529a392744e6c85617b&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:21.223514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a55797-8664-446e-bf50-e778335ef4c3 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:21.326119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a55797-8664-446e-bf50-e778335ef4c3 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:21.351967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:21.359847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f68e7-bd0f-4ddd-8061-7f9764fa90f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:21.360459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f68e7-bd0f-4ddd-8061-7f9764fa90f2 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:21.360848 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f68e7-bd0f-4ddd-8061-7f9764fa90f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:21.361144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f68e7-bd0f-4ddd-8061-7f9764fa90f2 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:21.361738 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f68e7-bd0f-4ddd-8061-7f9764fa90f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:21.363120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f68e7-bd0f-4ddd-8061-7f9764fa90f2 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:21.373923 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 455/925] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 12:38:21 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:38:22.096329 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022d2eb-e712-4c67-a88f-a93e8e8ba443 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:22.096865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022d2eb-e712-4c67-a88f-a93e8e8ba443 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:22.096865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022d2eb-e712-4c67-a88f-a93e8e8ba443 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:22.098429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 471/926] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:22.105655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df679ac8-efc7-4481-a1e7-4bb10374c0e4 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:22.105975 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df679ac8-efc7-4481-a1e7-4bb10374c0e4 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:22.106217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df679ac8-efc7-4481-a1e7-4bb10374c0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:22.126161 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-df679ac8-efc7-4481-a1e7-4bb10374c0e4 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:22.127379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-df679ac8-efc7-4481-a1e7-4bb10374c0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:22.197238 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 456/927] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:22.211354 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe63ed-9868-4fee-8372-e0702aeb9e91 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:22.211676 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe63ed-9868-4fee-8372-e0702aeb9e91 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:22.211952 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe63ed-9868-4fee-8372-e0702aeb9e91 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:22.213600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 472/928] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:22.218654 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d72002-7aac-473d-86b8-f76cc39e7685 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:22.218951 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d72002-7aac-473d-86b8-f76cc39e7685 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:22.219227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d72002-7aac-473d-86b8-f76cc39e7685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:22.237684 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-52d72002-7aac-473d-86b8-f76cc39e7685 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:22.238895 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-52d72002-7aac-473d-86b8-f76cc39e7685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:22.309636 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 457/929] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:22.478197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe63ed-9868-4fee-8372-e0702aeb9e91 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:22.580267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:22.589885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe63ed-9868-4fee-8372-e0702aeb9e91 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:22.602361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d72002-7aac-473d-86b8-f76cc39e7685 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:22.630389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8eaed-6ea0-4a3a-8166-f875d83cfb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:22.631067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8eaed-6ea0-4a3a-8166-f875d83cfb1e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:22.631375 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8eaed-6ea0-4a3a-8166-f875d83cfb1e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:22.632770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8eaed-6ea0-4a3a-8166-f875d83cfb1e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:22.632770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8eaed-6ea0-4a3a-8166-f875d83cfb1e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:22.634860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8eaed-6ea0-4a3a-8166-f875d83cfb1e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-14a8eaed-6ea0-4a3a-8166-f875d83cfb1e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:22.647528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:22.653914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:22.654873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 473/930] 10.222.0.46 () {56 vars in 1132 bytes} [Fri Sep 9 12:38:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:22.656152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8eaed-6ea0-4a3a-8166-f875d83cfb1e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:22.672307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:22.729604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d72002-7aac-473d-86b8-f76cc39e7685 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:22.767525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8eaed-6ea0-4a3a-8166-f875d83cfb1e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:22.774455 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac01fa-b018-49b6-831a-7931f4bfd0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:22.775174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac01fa-b018-49b6-831a-7931f4bfd0ea None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:22.775474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac01fa-b018-49b6-831a-7931f4bfd0ea None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:22.775830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac01fa-b018-49b6-831a-7931f4bfd0ea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:22.792563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac01fa-b018-49b6-831a-7931f4bfd0ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-26ac01fa-b018-49b6-831a-7931f4bfd0ea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:22.799745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:22.800808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:22.804072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:22.804072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:22.804072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:22.804072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:22.804072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:22.804072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:22.804072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:22.804072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:22.804072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:22.804072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:22.804072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:38:22.804072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:22.804072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 458/931] 10.222.0.46 () {56 vars in 1120 bytes} [Fri Sep 9 12:38:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:22.810082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac01fa-b018-49b6-831a-7931f4bfd0ea None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:22.824998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bbe69-f147-4392-86b9-6d1a65b5b66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:22.825938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bbe69-f147-4392-86b9-6d1a65b5b66e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:22.826376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bbe69-f147-4392-86b9-6d1a65b5b66e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:22.827023 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bbe69-f147-4392-86b9-6d1a65b5b66e None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:22.827561 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bbe69-f147-4392-86b9-6d1a65b5b66e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:22.827930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bbe69-f147-4392-86b9-6d1a65b5b66e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:22.829352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bbe69-f147-4392-86b9-6d1a65b5b66e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:22.840869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 474/932] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 12:38:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:22.841890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bbe69-f147-4392-86b9-6d1a65b5b66e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:22.872870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:22.941068 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac01fa-b018-49b6-831a-7931f4bfd0ea None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:22.954586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:22.957470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bbe69-f147-4392-86b9-6d1a65b5b66e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:22.976507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb93f2-36f1-4acf-9917-8f4a07862d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:22.977162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb93f2-36f1-4acf-9917-8f4a07862d9e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:22.977458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb93f2-36f1-4acf-9917-8f4a07862d9e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:22.977748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb93f2-36f1-4acf-9917-8f4a07862d9e None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:22.978413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb93f2-36f1-4acf-9917-8f4a07862d9e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:22.978746 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb93f2-36f1-4acf-9917-8f4a07862d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:22.980163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb93f2-36f1-4acf-9917-8f4a07862d9e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:22.989570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 459/933] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 12:38:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:22.990840 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37869b-3234-4124-90f7-e73f0a2c9308 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:22.991627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37869b-3234-4124-90f7-e73f0a2c9308 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:22.991902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37869b-3234-4124-90f7-e73f0a2c9308 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:22.993227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 460/934] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:22.996012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37869b-3234-4124-90f7-e73f0a2c9308 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:23.000558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412749a-f7d9-4bb7-a853-7ddf2a72c861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:23.001172 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412749a-f7d9-4bb7-a853-7ddf2a72c861 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:23.001447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412749a-f7d9-4bb7-a853-7ddf2a72c861 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:23.001713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412749a-f7d9-4bb7-a853-7ddf2a72c861 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:23.002344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412749a-f7d9-4bb7-a853-7ddf2a72c861 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:23.002678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412749a-f7d9-4bb7-a853-7ddf2a72c861 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:23.004023 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412749a-f7d9-4bb7-a853-7ddf2a72c861 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:23.009746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 475/935] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 12:38:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:23.010797 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3bd5c6-28bf-4994-895d-0051d81805da None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:23.011075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3bd5c6-28bf-4994-895d-0051d81805da None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:23.011348 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3bd5c6-28bf-4994-895d-0051d81805da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:23.031018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-4a3bd5c6-28bf-4994-895d-0051d81805da None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:23.032199 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4a3bd5c6-28bf-4994-895d-0051d81805da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:23.103701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 476/936] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:23.105253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3bd5c6-28bf-4994-895d-0051d81805da None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:23.118754 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37869b-3234-4124-90f7-e73f0a2c9308 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:23.138205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:23.161906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671c7a9-fffb-4d0b-9782-344ff101650c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:23.163255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671c7a9-fffb-4d0b-9782-344ff101650c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:23.163634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671c7a9-fffb-4d0b-9782-344ff101650c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:23.164010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671c7a9-fffb-4d0b-9782-344ff101650c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:23.179330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671c7a9-fffb-4d0b-9782-344ff101650c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:23.181893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671c7a9-fffb-4d0b-9782-344ff101650c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-f671c7a9-fffb-4d0b-9782-344ff101650c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:23.202125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:23.203370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:23.204480 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:23.204480 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:23.204480 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:23.206638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 461/937] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:23.212356 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671c7a9-fffb-4d0b-9782-344ff101650c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:23.233854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3bd5c6-28bf-4994-895d-0051d81805da None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:23.252631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:23.280831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52283ffa-43b3-4ce7-a7c2-4550f0c2f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:23.281548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52283ffa-43b3-4ce7-a7c2-4550f0c2f1d8 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:23.282029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52283ffa-43b3-4ce7-a7c2-4550f0c2f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:23.282349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52283ffa-43b3-4ce7-a7c2-4550f0c2f1d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:23.283652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52283ffa-43b3-4ce7-a7c2-4550f0c2f1d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:23.285390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52283ffa-43b3-4ce7-a7c2-4550f0c2f1d8 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:23.334940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671c7a9-fffb-4d0b-9782-344ff101650c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:23.371860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160329b-5088-4ac9-a119-02a9e364c1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:23.372789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160329b-5088-4ac9-a119-02a9e364c1bf None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:23.373264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160329b-5088-4ac9-a119-02a9e364c1bf None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:23.373714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160329b-5088-4ac9-a119-02a9e364c1bf None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:23.374542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160329b-5088-4ac9-a119-02a9e364c1bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:23.375020 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160329b-5088-4ac9-a119-02a9e364c1bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:23.376660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160329b-5088-4ac9-a119-02a9e364c1bf None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:23.386628 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 477/938] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:38:23 2022] POST /identity/v3/endpoints => generated 347 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:38:23.395177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 462/939] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:23.398229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52283ffa-43b3-4ce7-a7c2-4550f0c2f1d8 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:23.410899 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160329b-5088-4ac9-a119-02a9e364c1bf None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:23.499682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:23.523100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:23.546122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160329b-5088-4ac9-a119-02a9e364c1bf None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:23.551169 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52283ffa-43b3-4ce7-a7c2-4550f0c2f1d8 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:23.597622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f23368-ab30-4dc5-8100-6ae26eee935d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:23.599943 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23368-ab30-4dc5-8100-6ae26eee935d None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:23.600684 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23368-ab30-4dc5-8100-6ae26eee935d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:23.601363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23368-ab30-4dc5-8100-6ae26eee935d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:23.610611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f23368-ab30-4dc5-8100-6ae26eee935d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:23.612259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f23368-ab30-4dc5-8100-6ae26eee935d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:23.619656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2d9f7-eb8a-41dd-9a5c-17c8cbc1f728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:23.620266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2d9f7-eb8a-41dd-9a5c-17c8cbc1f728 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:23.620539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2d9f7-eb8a-41dd-9a5c-17c8cbc1f728 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:23.620809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2d9f7-eb8a-41dd-9a5c-17c8cbc1f728 None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:23.621469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2d9f7-eb8a-41dd-9a5c-17c8cbc1f728 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:23.622906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2d9f7-eb8a-41dd-9a5c-17c8cbc1f728 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:23.625522 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: WARNING keystone.server.flask.common [None req-c7f2d9f7-eb8a-41dd-9a5c-17c8cbc1f728 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-b0f23368-ab30-4dc5-8100-6ae26eee935d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:23.629917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:23.630974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:23.632265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:23.632265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:23.632265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:23.632265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:23.632265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:23.632265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:23.632265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:23.632265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:23.632265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:38:23.632265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:23.632265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 463/940] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 12:38:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:23.632874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35129f3-09d8-40d3-81a6-bdd0e089c41d None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:23.632874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35129f3-09d8-40d3-81a6-bdd0e089c41d None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:23.633327 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35129f3-09d8-40d3-81a6-bdd0e089c41d None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:23.635436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 464/941] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:23.638235 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35129f3-09d8-40d3-81a6-bdd0e089c41d None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:23.673849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.sql.core [None req-c7f2d9f7-eb8a-41dd-9a5c-17c8cbc1f728 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 12:38:23.673849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 12:38:23.673849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [parameters: {'id': '7be2e461795845f4a3b0e91f2c17465d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 12:38:23.673849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88041) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 12:38:23.680795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-c7f2d9f7-eb8a-41dd-9a5c-17c8cbc1f728 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7be2e461795845f4a3b0e91f2c17465d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7be2e461795845f4a3b0e91f2c17465d. Sep 09 12:38:23.680795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:23.680795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 12:38:23.680795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 12:38:23.680795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 12:38:23.680795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 12:38:23.680795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 12:38:23.680795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 12:38:23.680795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 12:38:23.680795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application conn.query(q) Sep 09 12:38:23.680795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 12:38:23.680795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 12:38:23.680795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application result.read() Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 12:38:23.681686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application next(self.gen) Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application yield resource Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application next(self.gen) Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise self.value Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application session.commit() Sep 09 12:38:23.682744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.session.flush() Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application compat.raise_( Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 12:38:23.683666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 12:38:23.684591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application conn.query(q) Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application result.read() Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 12:38:23.685402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [parameters: {'id': '7be2e461795845f4a3b0e91f2c17465d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:23.688680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:23.691352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:23.699682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:23.699682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 12:38:23.699682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 12:38:23.699682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:23.699682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:23.699682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 12:38:23.699682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 12:38:23.699682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7be2e461795845f4a3b0e91f2c17465d. Sep 09 12:38:23.699682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:23.699682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 478/942] 10.222.0.46 () {60 vars in 1140 bytes} [Fri Sep 9 12:38:23 2022] POST /identity/v3/projects => generated 262 bytes in 285 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:23.699682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38da2fe-eefe-4061-92a6-27a37b2fcf85 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:23.699682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38da2fe-eefe-4061-92a6-27a37b2fcf85 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:23.700900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38da2fe-eefe-4061-92a6-27a37b2fcf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:23.706258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d38da2fe-eefe-4061-92a6-27a37b2fcf85 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:23.707933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d38da2fe-eefe-4061-92a6-27a37b2fcf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:23.764353 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35129f3-09d8-40d3-81a6-bdd0e089c41d None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:23.781265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 479/943] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:23.791441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38da2fe-eefe-4061-92a6-27a37b2fcf85 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:23.801848 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badb270-1406-4340-8649-18d93268a50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:23.802180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badb270-1406-4340-8649-18d93268a50f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:23.802473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badb270-1406-4340-8649-18d93268a50f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:23.802781 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badb270-1406-4340-8649-18d93268a50f None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:23.804026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badb270-1406-4340-8649-18d93268a50f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:23.804337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badb270-1406-4340-8649-18d93268a50f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:23.806022 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badb270-1406-4340-8649-18d93268a50f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:23.830953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 465/944] 10.222.0.46 () {56 vars in 1149 bytes} [Fri Sep 9 12:38:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:23.831840 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badb270-1406-4340-8649-18d93268a50f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:23.855878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:23.862845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:23.909558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38da2fe-eefe-4061-92a6-27a37b2fcf85 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:23.942951 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badb270-1406-4340-8649-18d93268a50f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:23.953915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a634f4-b4f7-48aa-be9d-216cfe296f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:23.954584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a634f4-b4f7-48aa-be9d-216cfe296f34 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:23.954859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a634f4-b4f7-48aa-be9d-216cfe296f34 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:23.955127 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a634f4-b4f7-48aa-be9d-216cfe296f34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:23.961976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a634f4-b4f7-48aa-be9d-216cfe296f34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:23.963629 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a634f4-b4f7-48aa-be9d-216cfe296f34 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-80a634f4-b4f7-48aa-be9d-216cfe296f34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:23.976391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:23.977476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:23.978604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:23.978604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:23.978604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:23.978604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:23.978604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:23.978604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:23.978604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:23.978604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:23.978604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 480/945] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:23.984752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a634f4-b4f7-48aa-be9d-216cfe296f34 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:23.992930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72383588-c812-4d9c-813f-58fdaa6c5666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:23.993701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72383588-c812-4d9c-813f-58fdaa6c5666 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:23.994132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72383588-c812-4d9c-813f-58fdaa6c5666 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:23.994550 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72383588-c812-4d9c-813f-58fdaa6c5666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:24.008618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72383588-c812-4d9c-813f-58fdaa6c5666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:24.010675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72383588-c812-4d9c-813f-58fdaa6c5666 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-72383588-c812-4d9c-813f-58fdaa6c5666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:24.030979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:24.032125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:24.033249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:24.033249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:24.033249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:24.038588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 466/946] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:24.040898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72383588-c812-4d9c-813f-58fdaa6c5666 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:24.104492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a634f4-b4f7-48aa-be9d-216cfe296f34 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:24.109502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:24.125626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:24.147831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0edd7-0a22-40fe-9743-ea0f7c8d9b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:24.148433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0edd7-0a22-40fe-9743-ea0f7c8d9b42 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:24.148714 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0edd7-0a22-40fe-9743-ea0f7c8d9b42 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:24.148987 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0edd7-0a22-40fe-9743-ea0f7c8d9b42 None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:24.149650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0edd7-0a22-40fe-9743-ea0f7c8d9b42 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:24.149955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0edd7-0a22-40fe-9743-ea0f7c8d9b42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:24.151398 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0edd7-0a22-40fe-9743-ea0f7c8d9b42 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:24.158798 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72383588-c812-4d9c-813f-58fdaa6c5666 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:24.173530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 481/947] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 12:38:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:24.174436 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0edd7-0a22-40fe-9743-ea0f7c8d9b42 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:24.208155 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d5c4d-41ff-4743-b6e8-e75a9d995c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:24.209315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d5c4d-41ff-4743-b6e8-e75a9d995c92 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:24.210791 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d5c4d-41ff-4743-b6e8-e75a9d995c92 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:24.211252 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d5c4d-41ff-4743-b6e8-e75a9d995c92 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:24.212047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d5c4d-41ff-4743-b6e8-e75a9d995c92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:24.212456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d5c4d-41ff-4743-b6e8-e75a9d995c92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:24.214298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d5c4d-41ff-4743-b6e8-e75a9d995c92 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:24.232898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 467/948] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:24.239698 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d5c4d-41ff-4743-b6e8-e75a9d995c92 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:24.305946 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0edd7-0a22-40fe-9743-ea0f7c8d9b42 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:24.345864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76d14c-8b07-40f2-b0ba-df056c539095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:24.346908 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76d14c-8b07-40f2-b0ba-df056c539095 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:24.347380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76d14c-8b07-40f2-b0ba-df056c539095 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:24.347826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76d14c-8b07-40f2-b0ba-df056c539095 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:24.348695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76d14c-8b07-40f2-b0ba-df056c539095 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:24.350977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76d14c-8b07-40f2-b0ba-df056c539095 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:24.360117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d5c4d-41ff-4743-b6e8-e75a9d995c92 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-9e76d14c-8b07-40f2-b0ba-df056c539095 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:24.362017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:24.363067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:24.363914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:24.372552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 482/949] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 12:38:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:24.374288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76d14c-8b07-40f2-b0ba-df056c539095 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:24.380748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:24.401353 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e39a4c-2ab2-4bcb-a4b0-ba5f56ccf220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:24.402037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e39a4c-2ab2-4bcb-a4b0-ba5f56ccf220 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:24.402326 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e39a4c-2ab2-4bcb-a4b0-ba5f56ccf220 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:24.402630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e39a4c-2ab2-4bcb-a4b0-ba5f56ccf220 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:24.410457 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e39a4c-2ab2-4bcb-a4b0-ba5f56ccf220 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:24.411914 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e39a4c-2ab2-4bcb-a4b0-ba5f56ccf220 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:24.415174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-d2e39a4c-2ab2-4bcb-a4b0-ba5f56ccf220 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:24.425217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:24.432242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:24.433409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:24.433409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:24.433409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:24.433409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:24.433409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:24.433409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:24.433409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:24.433409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:24.433409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 468/950] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:24.436930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e39a4c-2ab2-4bcb-a4b0-ba5f56ccf220 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:24.491941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76d14c-8b07-40f2-b0ba-df056c539095 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:24.535982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57787fd0-efbf-410b-a148-825ad6025172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:24.537135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57787fd0-efbf-410b-a148-825ad6025172 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:24.537514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57787fd0-efbf-410b-a148-825ad6025172 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:24.538686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57787fd0-efbf-410b-a148-825ad6025172 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:24.539917 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57787fd0-efbf-410b-a148-825ad6025172 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:24.540319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57787fd0-efbf-410b-a148-825ad6025172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:24.542787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57787fd0-efbf-410b-a148-825ad6025172 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:24.554529 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 483/951] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 12:38:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:24.559849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57787fd0-efbf-410b-a148-825ad6025172 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:24.564001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e39a4c-2ab2-4bcb-a4b0-ba5f56ccf220 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:24.613788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72a95d-2d2f-4acf-83dd-f1e73e0177ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:24.615020 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72a95d-2d2f-4acf-83dd-f1e73e0177ed None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:24.615437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72a95d-2d2f-4acf-83dd-f1e73e0177ed None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:24.615868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72a95d-2d2f-4acf-83dd-f1e73e0177ed None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:24.616701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72a95d-2d2f-4acf-83dd-f1e73e0177ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:24.617103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72a95d-2d2f-4acf-83dd-f1e73e0177ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:24.619366 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72a95d-2d2f-4acf-83dd-f1e73e0177ed None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:24.642563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 469/952] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:38:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:24.644599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72a95d-2d2f-4acf-83dd-f1e73e0177ed None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:24.681717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:24.685062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57787fd0-efbf-410b-a148-825ad6025172 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:24.692471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:24.724154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15c5c7-6ea8-45bf-92eb-1396b260cd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:24.725933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15c5c7-6ea8-45bf-92eb-1396b260cd15 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:24.726749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15c5c7-6ea8-45bf-92eb-1396b260cd15 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:24.727320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15c5c7-6ea8-45bf-92eb-1396b260cd15 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:24.728599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15c5c7-6ea8-45bf-92eb-1396b260cd15 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:24.729197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15c5c7-6ea8-45bf-92eb-1396b260cd15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:24.730747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15c5c7-6ea8-45bf-92eb-1396b260cd15 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:24.768097 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 484/953] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 12:38:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:24.769926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15c5c7-6ea8-45bf-92eb-1396b260cd15 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:24.771202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72a95d-2d2f-4acf-83dd-f1e73e0177ed None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:24.820549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b481ad2-8de5-4fce-bebf-306ef4587813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:24.821293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b481ad2-8de5-4fce-bebf-306ef4587813 None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:24.821669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b481ad2-8de5-4fce-bebf-306ef4587813 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:24.822043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b481ad2-8de5-4fce-bebf-306ef4587813 None admin] PATH_INFO: `/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/ec77f4e9939b4dd685b850f8b6fb1e56/roles/0f979abba91c44569a1ceb70bceceacb` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:24.825034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b481ad2-8de5-4fce-bebf-306ef4587813 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e2dc716277d42ae8316ebb2d36293a8, user_id=ec77f4e9939b4dd685b850f8b6fb1e56, role_id=0f979abba91c44569a1ceb70bceceacb)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:24.827441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b481ad2-8de5-4fce-bebf-306ef4587813 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:24.902112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15c5c7-6ea8-45bf-92eb-1396b260cd15 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:24.916003 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 470/954] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:38:24 2022] PUT /identity/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/ec77f4e9939b4dd685b850f8b6fb1e56/roles/0f979abba91c44569a1ceb70bceceacb => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:38:24.953827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd50c97-a74b-4313-a8a0-dcc85b68bf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:24.954659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd50c97-a74b-4313-a8a0-dcc85b68bf78 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:24.955086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd50c97-a74b-4313-a8a0-dcc85b68bf78 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:24.955427 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd50c97-a74b-4313-a8a0-dcc85b68bf78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:24.956270 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd50c97-a74b-4313-a8a0-dcc85b68bf78 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:24.957763 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd50c97-a74b-4313-a8a0-dcc85b68bf78 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:25.038670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 485/955] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:38:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:38:25.658532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410fd76-7869-4cb5-a7e4-c935d1e279ee None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:25.659083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410fd76-7869-4cb5-a7e4-c935d1e279ee None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:25.659083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410fd76-7869-4cb5-a7e4-c935d1e279ee None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:25.660423 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 471/956] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:25.666380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458de65-885b-4b2c-a298-35117d5340de None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:25.666905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458de65-885b-4b2c-a298-35117d5340de None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:25.667063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458de65-885b-4b2c-a298-35117d5340de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:25.686159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-2458de65-885b-4b2c-a298-35117d5340de None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:25.687376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2458de65-885b-4b2c-a298-35117d5340de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:25.701890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:25.766624 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 486/957] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:26.035481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410fd76-7869-4cb5-a7e4-c935d1e279ee None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:26.135003 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:26.141794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410fd76-7869-4cb5-a7e4-c935d1e279ee None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:26.177440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671ddb0-8c30-42f0-af78-59ae7816bdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:26.178083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671ddb0-8c30-42f0-af78-59ae7816bdb9 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:26.178541 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671ddb0-8c30-42f0-af78-59ae7816bdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:26.178823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671ddb0-8c30-42f0-af78-59ae7816bdb9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:26.179409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671ddb0-8c30-42f0-af78-59ae7816bdb9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:26.180767 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671ddb0-8c30-42f0-af78-59ae7816bdb9 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-9671ddb0-8c30-42f0-af78-59ae7816bdb9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:26.189956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:26.192536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:26.195163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 472/958] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:26.199575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458de65-885b-4b2c-a298-35117d5340de None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:26.305539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458de65-885b-4b2c-a298-35117d5340de None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:26.341128 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315c455-1b43-485d-b982-7c682d6b6e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:26.341736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315c455-1b43-485d-b982-7c682d6b6e5a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:26.342012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315c455-1b43-485d-b982-7c682d6b6e5a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:26.342280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315c455-1b43-485d-b982-7c682d6b6e5a None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:26.342932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315c455-1b43-485d-b982-7c682d6b6e5a None admin] RBAC: Adding query filter params (name=compute) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:26.343229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315c455-1b43-485d-b982-7c682d6b6e5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:26.344569 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315c455-1b43-485d-b982-7c682d6b6e5a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:26.349826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 487/959] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 12:38:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:26.355776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671ddb0-8c30-42f0-af78-59ae7816bdb9 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:26.464813 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671ddb0-8c30-42f0-af78-59ae7816bdb9 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:26.502158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e2aa1-4ac6-4995-883d-fa876b981d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:26.503134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e2aa1-4ac6-4995-883d-fa876b981d4d None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:26.503624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e2aa1-4ac6-4995-883d-fa876b981d4d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:26.504139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e2aa1-4ac6-4995-883d-fa876b981d4d None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:26.504971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e2aa1-4ac6-4995-883d-fa876b981d4d None admin] RBAC: Adding query filter params (type=compute) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:26.505474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e2aa1-4ac6-4995-883d-fa876b981d4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:26.507078 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e2aa1-4ac6-4995-883d-fa876b981d4d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:26.512265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 473/960] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 12:38:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:26.566071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3eda8-9490-4167-bceb-bf09df7bd614 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:26.566270 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3eda8-9490-4167-bceb-bf09df7bd614 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:26.566596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3eda8-9490-4167-bceb-bf09df7bd614 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:26.568317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 488/961] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:26.574388 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1b6c8-14e5-4402-96e8-919c03719eca None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:26.575047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1b6c8-14e5-4402-96e8-919c03719eca None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:26.575556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1b6c8-14e5-4402-96e8-919c03719eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:26.596329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-78a1b6c8-14e5-4402-96e8-919c03719eca None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:26.597905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-78a1b6c8-14e5-4402-96e8-919c03719eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:26.630372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:26.703702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 474/962] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:26.987438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3eda8-9490-4167-bceb-bf09df7bd614 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:27.000541 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:27.098257 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3eda8-9490-4167-bceb-bf09df7bd614 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:27.150752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fedcd9-28e2-4bb8-9526-67edefb253d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:27.151415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fedcd9-28e2-4bb8-9526-67edefb253d5 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:27.151697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fedcd9-28e2-4bb8-9526-67edefb253d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:27.151974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fedcd9-28e2-4bb8-9526-67edefb253d5 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:27.152619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fedcd9-28e2-4bb8-9526-67edefb253d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:27.154031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fedcd9-28e2-4bb8-9526-67edefb253d5 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:27.200130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.sql.core [None req-d1fedcd9-28e2-4bb8-9526-67edefb253d5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 12:38:27.200130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 12:38:27.200130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [parameters: {'user_id': '26216099115d47abbfd8a215a70a2c8d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 12:38:27.200130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88041) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-d1fedcd9-28e2-4bb8-9526-67edefb253d5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application conn.query(q) Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application result.read() Sep 09 12:38:27.207388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 12:38:27.211160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application next(self.gen) Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application yield resource Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application next(self.gen) Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise self.value Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application session.commit() Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 12:38:27.212370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.session.flush() Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application compat.raise_( Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 12:38:27.213653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 12:38:27.214995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application conn.query(q) Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application result.read() Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 12:38:27.216137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [parameters: {'user_id': '26216099115d47abbfd8a215a70a2c8d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:27.217255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 12:38:27.218098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 12:38:27.219295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 12:38:27.219295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 12:38:27.219295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 12:38:27.219295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 12:38:27.219295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 12:38:27.219295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:27.219295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 489/963] 10.222.0.46 () {60 vars in 1131 bytes} [Fri Sep 9 12:38:26 2022] POST /identity/v3/users => generated 160 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:27.219816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1b6c8-14e5-4402-96e8-919c03719eca None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:27.243862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981deec5-3c9c-4067-aef4-0dc909e9f44e None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:27.244044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981deec5-3c9c-4067-aef4-0dc909e9f44e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:27.244328 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981deec5-3c9c-4067-aef4-0dc909e9f44e None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:27.252664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 490/964] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:27.253884 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692667f2-4799-450e-8f4a-5d8f962fb23d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:27.254171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692667f2-4799-450e-8f4a-5d8f962fb23d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:27.254448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692667f2-4799-450e-8f4a-5d8f962fb23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:27.275869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-692667f2-4799-450e-8f4a-5d8f962fb23d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:27.277922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-692667f2-4799-450e-8f4a-5d8f962fb23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:27.327287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1b6c8-14e5-4402-96e8-919c03719eca None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:27.358536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 491/965] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:27.358536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af841d36-a8b5-4812-bc4a-9c5cfe6679b3 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:27.358536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af841d36-a8b5-4812-bc4a-9c5cfe6679b3 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:27.358536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af841d36-a8b5-4812-bc4a-9c5cfe6679b3 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:27.358536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 492/966] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:27.363974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a603890-0d82-48ec-8b6d-1358946f241d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:27.364259 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a603890-0d82-48ec-8b6d-1358946f241d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:27.364558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a603890-0d82-48ec-8b6d-1358946f241d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:27.367848 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2685b-4806-41dc-8460-0f3044072b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:27.368707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2685b-4806-41dc-8460-0f3044072b85 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:27.369127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2685b-4806-41dc-8460-0f3044072b85 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:27.369536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2685b-4806-41dc-8460-0f3044072b85 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:27.372615 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:27.377115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2685b-4806-41dc-8460-0f3044072b85 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:27.378653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2685b-4806-41dc-8460-0f3044072b85 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:27.383662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0a603890-0d82-48ec-8b6d-1358946f241d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:27.384857 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0a603890-0d82-48ec-8b6d-1358946f241d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-bbe2685b-4806-41dc-8460-0f3044072b85 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:27.390471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:27.391411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:27.392373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:27.392373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:27.392373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:27.392373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:27.392373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:27.392373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:27.392373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:27.392373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:27.392373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:38:27.392373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:27.392373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 475/967] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:38:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:27.403454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2685b-4806-41dc-8460-0f3044072b85 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:27.424302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:27.474951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 493/968] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:27.523208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2685b-4806-41dc-8460-0f3044072b85 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:27.557001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7c1a-77d4-40f4-9077-821b6bbd46bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:27.557610 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7c1a-77d4-40f4-9077-821b6bbd46bd None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:27.557891 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7c1a-77d4-40f4-9077-821b6bbd46bd None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:27.558162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7c1a-77d4-40f4-9077-821b6bbd46bd None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:27.558882 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba7c1a-77d4-40f4-9077-821b6bbd46bd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:27.559183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba7c1a-77d4-40f4-9077-821b6bbd46bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:27.560591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba7c1a-77d4-40f4-9077-821b6bbd46bd None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:27.595620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 476/969] 10.222.0.46 () {56 vars in 1101 bytes} [Fri Sep 9 12:38:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:27.649416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a603890-0d82-48ec-8b6d-1358946f241d None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:27.758748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7c1a-77d4-40f4-9077-821b6bbd46bd None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:27.767714 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a603890-0d82-48ec-8b6d-1358946f241d None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:27.800587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:27.806697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80ad56-c272-4694-9ebf-1f652382d34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:27.807532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80ad56-c272-4694-9ebf-1f652382d34d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:27.809342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80ad56-c272-4694-9ebf-1f652382d34d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:27.809342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80ad56-c272-4694-9ebf-1f652382d34d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:27.824944 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80ad56-c272-4694-9ebf-1f652382d34d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-eb80ad56-c272-4694-9ebf-1f652382d34d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:27.830044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:27.831027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:27.831872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:27.831872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:27.831872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:27.831872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:27.831872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:27.831872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:27.831872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:27.831872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:27.831872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:27.831872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:27.831872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:38:27.831872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:27.831872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 494/970] 10.222.0.46 () {56 vars in 1120 bytes} [Fri Sep 9 12:38:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:27.839229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80ad56-c272-4694-9ebf-1f652382d34d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:27.859493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:27.895746 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7c1a-77d4-40f4-9077-821b6bbd46bd None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:27.945187 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce277eba-7d14-48ad-8cfd-a125fc46ec2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:27.945834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce277eba-7d14-48ad-8cfd-a125fc46ec2a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:27.946117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce277eba-7d14-48ad-8cfd-a125fc46ec2a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:27.946782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce277eba-7d14-48ad-8cfd-a125fc46ec2a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:27.947378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce277eba-7d14-48ad-8cfd-a125fc46ec2a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:27.949083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce277eba-7d14-48ad-8cfd-a125fc46ec2a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-ce277eba-7d14-48ad-8cfd-a125fc46ec2a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:27.957847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:27.958897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:27.959973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 477/971] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 12:38:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:27.964970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce277eba-7d14-48ad-8cfd-a125fc46ec2a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:27.974242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80ad56-c272-4694-9ebf-1f652382d34d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:28.010091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f7448-168e-45bf-ab52-33a31aa67c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:28.010738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f7448-168e-45bf-ab52-33a31aa67c65 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:28.010985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f7448-168e-45bf-ab52-33a31aa67c65 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:28.011249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f7448-168e-45bf-ab52-33a31aa67c65 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:28.011910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f7448-168e-45bf-ab52-33a31aa67c65 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:28.012207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f7448-168e-45bf-ab52-33a31aa67c65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:28.013679 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f7448-168e-45bf-ab52-33a31aa67c65 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:28.022476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 495/972] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 12:38:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:28.028065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f7448-168e-45bf-ab52-33a31aa67c65 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:28.083615 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:28.086135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce277eba-7d14-48ad-8cfd-a125fc46ec2a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:28.135182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f5ef9-b895-4a2f-b29d-5cc02e0cff5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:28.135486 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f7448-168e-45bf-ab52-33a31aa67c65 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:28.135799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f5ef9-b895-4a2f-b29d-5cc02e0cff5e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:28.136082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f5ef9-b895-4a2f-b29d-5cc02e0cff5e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:28.136353 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f5ef9-b895-4a2f-b29d-5cc02e0cff5e None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:28.137144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f5ef9-b895-4a2f-b29d-5cc02e0cff5e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:28.137506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f5ef9-b895-4a2f-b29d-5cc02e0cff5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:28.139724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f5ef9-b895-4a2f-b29d-5cc02e0cff5e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:28.141832 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:28.146705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 478/973] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 12:38:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:28.152978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f5ef9-b895-4a2f-b29d-5cc02e0cff5e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:28.171316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd904f-c50e-490b-b6a0-22bf0264f23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:28.172083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd904f-c50e-490b-b6a0-22bf0264f23b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:28.172374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd904f-c50e-490b-b6a0-22bf0264f23b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:28.172642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd904f-c50e-490b-b6a0-22bf0264f23b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:28.186341 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd904f-c50e-490b-b6a0-22bf0264f23b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:28.187802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd904f-c50e-490b-b6a0-22bf0264f23b None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-d2bd904f-c50e-490b-b6a0-22bf0264f23b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:28.202235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:28.203245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:28.209424 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:28.209424 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:28.209424 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:28.209424 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 496/974] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:28.209424 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd904f-c50e-490b-b6a0-22bf0264f23b None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:28.270262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f5ef9-b895-4a2f-b29d-5cc02e0cff5e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:28.311227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a54d08-dd01-4fc8-a08a-7734f52e6ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:28.311842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a54d08-dd01-4fc8-a08a-7734f52e6ac0 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:28.312120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a54d08-dd01-4fc8-a08a-7734f52e6ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:28.312392 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a54d08-dd01-4fc8-a08a-7734f52e6ac0 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:28.313006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a54d08-dd01-4fc8-a08a-7734f52e6ac0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:28.313308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a54d08-dd01-4fc8-a08a-7734f52e6ac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:28.314680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a54d08-dd01-4fc8-a08a-7734f52e6ac0 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:28.320670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 479/975] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 12:38:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:28.326380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a54d08-dd01-4fc8-a08a-7734f52e6ac0 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:28.329523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd904f-c50e-490b-b6a0-22bf0264f23b None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:28.338208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:28.365581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513d1f3-7b71-4ac0-a1e4-d089fbf57d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:28.367087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513d1f3-7b71-4ac0-a1e4-d089fbf57d38 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:28.367423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513d1f3-7b71-4ac0-a1e4-d089fbf57d38 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:28.367698 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513d1f3-7b71-4ac0-a1e4-d089fbf57d38 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:28.368346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513d1f3-7b71-4ac0-a1e4-d089fbf57d38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:28.368649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513d1f3-7b71-4ac0-a1e4-d089fbf57d38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:28.370016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513d1f3-7b71-4ac0-a1e4-d089fbf57d38 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:28.384237 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 497/976] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:28.389491 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513d1f3-7b71-4ac0-a1e4-d089fbf57d38 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:28.394192 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:28.435252 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a54d08-dd01-4fc8-a08a-7734f52e6ac0 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:28.470730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f40dd-1ecb-41a7-ae8b-713d3da54088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:28.471348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f40dd-1ecb-41a7-ae8b-713d3da54088 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:28.471618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f40dd-1ecb-41a7-ae8b-713d3da54088 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:28.471886 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f40dd-1ecb-41a7-ae8b-713d3da54088 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:28.472992 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f40dd-1ecb-41a7-ae8b-713d3da54088 None admin] RBAC: Adding query filter params (interface=admin, service_id=598c6f15c2b14529a392744e6c85617b, region_id=RegionOne) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:28.473303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f40dd-1ecb-41a7-ae8b-713d3da54088 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:28.475001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f40dd-1ecb-41a7-ae8b-713d3da54088 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:28.485557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 480/977] 10.222.0.46 () {56 vars in 1249 bytes} [Fri Sep 9 12:38:28 2022] GET /identity/v3/endpoints?service_id=598c6f15c2b14529a392744e6c85617b&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:28.486409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f40dd-1ecb-41a7-ae8b-713d3da54088 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:28.499098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513d1f3-7b71-4ac0-a1e4-d089fbf57d38 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:28.535901 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c2ae9-be05-4c76-aed4-c35d0464c61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:28.536528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c2ae9-be05-4c76-aed4-c35d0464c61a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:28.536836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c2ae9-be05-4c76-aed4-c35d0464c61a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:28.537109 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c2ae9-be05-4c76-aed4-c35d0464c61a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:28.544458 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c2ae9-be05-4c76-aed4-c35d0464c61a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:28.545870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c2ae9-be05-4c76-aed4-c35d0464c61a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-0f3c2ae9-be05-4c76-aed4-c35d0464c61a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:28.558187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:28.559363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:28.560479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:28.560479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:28.560479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:28.560479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:28.560479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:28.560479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:28.560479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:28.560479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:28.560479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:38:28.560479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:28.562575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 498/978] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 12:38:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:28.568214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c2ae9-be05-4c76-aed4-c35d0464c61a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:28.602037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:28.610052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f40dd-1ecb-41a7-ae8b-713d3da54088 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:28.641287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:28.651404 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace85c4-d204-4396-b3e0-2ed4b97d8316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:28.652933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace85c4-d204-4396-b3e0-2ed4b97d8316 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:28.652933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace85c4-d204-4396-b3e0-2ed4b97d8316 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:28.652933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace85c4-d204-4396-b3e0-2ed4b97d8316 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:28.653340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace85c4-d204-4396-b3e0-2ed4b97d8316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:28.654798 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace85c4-d204-4396-b3e0-2ed4b97d8316 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:28.671000 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 481/979] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 12:38:28 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:38:28.681998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c2ae9-be05-4c76-aed4-c35d0464c61a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:28.721878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efbab1-c0e6-4077-ab5c-560c035cae40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:28.722603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efbab1-c0e6-4077-ab5c-560c035cae40 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:28.722905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efbab1-c0e6-4077-ab5c-560c035cae40 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:28.723136 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efbab1-c0e6-4077-ab5c-560c035cae40 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:28.723912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efbab1-c0e6-4077-ab5c-560c035cae40 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:28.724211 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efbab1-c0e6-4077-ab5c-560c035cae40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:28.725916 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efbab1-c0e6-4077-ab5c-560c035cae40 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:28.746967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 499/980] 10.222.0.46 () {56 vars in 1149 bytes} [Fri Sep 9 12:38:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:28.752700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace85c4-d204-4396-b3e0-2ed4b97d8316 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:28.759398 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeea61-e70c-4962-930c-3988f639c25b None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:28.759707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeea61-e70c-4962-930c-3988f639c25b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:28.759961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeea61-e70c-4962-930c-3988f639c25b None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:28.761622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 500/981] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:38:28.767470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6823a60-5608-4edc-9e0f-89dfa7c5f463 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:28.767798 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6823a60-5608-4edc-9e0f-89dfa7c5f463 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:28.768080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6823a60-5608-4edc-9e0f-89dfa7c5f463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:28.788844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-e6823a60-5608-4edc-9e0f-89dfa7c5f463 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:28.790089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e6823a60-5608-4edc-9e0f-89dfa7c5f463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:28.863040 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace85c4-d204-4396-b3e0-2ed4b97d8316 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:28.870381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 501/982] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:28.906051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae37b2-e3e0-45f4-93cd-8572861bf9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:28.906685 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae37b2-e3e0-45f4-93cd-8572861bf9f1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:28.906963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae37b2-e3e0-45f4-93cd-8572861bf9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:28.907228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae37b2-e3e0-45f4-93cd-8572861bf9f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:28.986411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:28.997010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae37b2-e3e0-45f4-93cd-8572861bf9f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:28.998535 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae37b2-e3e0-45f4-93cd-8572861bf9f1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:29.097556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 482/983] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:38:28 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:38:29.103458 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6823a60-5608-4edc-9e0f-89dfa7c5f463 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:29.122426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:29.144690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae37b2-e3e0-45f4-93cd-8572861bf9f1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:29.212064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6823a60-5608-4edc-9e0f-89dfa7c5f463 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:29.242564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:29.249359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d4c6b-50e5-4bc7-a45a-cc8bc18a1acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:29.249933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d4c6b-50e5-4bc7-a45a-cc8bc18a1acb None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:29.250205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d4c6b-50e5-4bc7-a45a-cc8bc18a1acb None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:29.250474 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d4c6b-50e5-4bc7-a45a-cc8bc18a1acb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:29.251934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae37b2-e3e0-45f4-93cd-8572861bf9f1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:29.261419 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d4c6b-50e5-4bc7-a45a-cc8bc18a1acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:29.262922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d4c6b-50e5-4bc7-a45a-cc8bc18a1acb None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-fc9d4c6b-50e5-4bc7-a45a-cc8bc18a1acb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:29.279306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:29.280297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:29.282041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:29.282041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:29.282041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:29.282041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 502/984] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:29.285829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d4c6b-50e5-4bc7-a45a-cc8bc18a1acb None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:29.290138 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb98862-0c23-4083-bd5f-5c51d0083544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:29.290784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98862-0c23-4083-bd5f-5c51d0083544 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:29.291067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98862-0c23-4083-bd5f-5c51d0083544 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:29.291340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98862-0c23-4083-bd5f-5c51d0083544 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:29.291986 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb98862-0c23-4083-bd5f-5c51d0083544 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:29.293412 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb98862-0c23-4083-bd5f-5c51d0083544 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:29.349790 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:29.377198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 483/985] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:38:29 2022] POST /identity/v3/services => generated 242 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:38:29.385920 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d4c6b-50e5-4bc7-a45a-cc8bc18a1acb None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:29.433335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfb91c-a2db-4565-b151-c97eaedd6ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:29.433966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfb91c-a2db-4565-b151-c97eaedd6ff5 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:29.434286 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfb91c-a2db-4565-b151-c97eaedd6ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:29.434584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfb91c-a2db-4565-b151-c97eaedd6ff5 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:29.435305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bfb91c-a2db-4565-b151-c97eaedd6ff5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:29.435603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bfb91c-a2db-4565-b151-c97eaedd6ff5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:29.437333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bfb91c-a2db-4565-b151-c97eaedd6ff5 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:29.455924 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 503/986] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:29.457560 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb98862-0c23-4083-bd5f-5c51d0083544 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:29.568343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb98862-0c23-4083-bd5f-5c51d0083544 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:29.607076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f600f7d-96fa-429f-abbb-78042da12388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:29.607725 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f600f7d-96fa-429f-abbb-78042da12388 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:29.608002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f600f7d-96fa-429f-abbb-78042da12388 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:29.608270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f600f7d-96fa-429f-abbb-78042da12388 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:29.615520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f600f7d-96fa-429f-abbb-78042da12388 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:29.616985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f600f7d-96fa-429f-abbb-78042da12388 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-7f600f7d-96fa-429f-abbb-78042da12388 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:29.628296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:29.629246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:29.630910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:29.630910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:29.630910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:29.630910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:29.630910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:29.630910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:29.630910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:29.630910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:29.630910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 484/987] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:29.635321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfb91c-a2db-4565-b151-c97eaedd6ff5 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:29.737375 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfb91c-a2db-4565-b151-c97eaedd6ff5 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:29.760046 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:29.771560 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8862add-6662-449f-b751-2d2120b3862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:29.772932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8862add-6662-449f-b751-2d2120b3862e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:29.773217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8862add-6662-449f-b751-2d2120b3862e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:29.773490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8862add-6662-449f-b751-2d2120b3862e None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:29.774183 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8862add-6662-449f-b751-2d2120b3862e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:29.774506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8862add-6662-449f-b751-2d2120b3862e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:29.777795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8862add-6662-449f-b751-2d2120b3862e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:29.793197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 504/988] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:38:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:29.794470 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f600f7d-96fa-429f-abbb-78042da12388 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:29.810127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:29.898895 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f600f7d-96fa-429f-abbb-78042da12388 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:29.931680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323dcc9-d822-4933-8e3d-51631a29a27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:29.932288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323dcc9-d822-4933-8e3d-51631a29a27c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:29.932642 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323dcc9-d822-4933-8e3d-51631a29a27c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:29.933098 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323dcc9-d822-4933-8e3d-51631a29a27c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:29.933728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323dcc9-d822-4933-8e3d-51631a29a27c None admin] RBAC: Adding query filter params (role.id=0f979abba91c44569a1ceb70bceceacb, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=ec77f4e9939b4dd685b850f8b6fb1e56) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:29.933844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323dcc9-d822-4933-8e3d-51631a29a27c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:29.935298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323dcc9-d822-4933-8e3d-51631a29a27c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:29.941743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 485/989] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:38:29 2022] GET /identity/v3/role_assignments?user.id=ec77f4e9939b4dd685b850f8b6fb1e56&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=0f979abba91c44569a1ceb70bceceacb => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:30.948567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdce6fb-8214-4a3e-b5f0-93168f19d449 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:30.949090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdce6fb-8214-4a3e-b5f0-93168f19d449 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:30.949090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdce6fb-8214-4a3e-b5f0-93168f19d449 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:30.950509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 505/990] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:30.955596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503ddbb-9da8-4a48-8ae4-7f49da3525f2 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:30.955966 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503ddbb-9da8-4a48-8ae4-7f49da3525f2 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:30.956154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503ddbb-9da8-4a48-8ae4-7f49da3525f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:30.976583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-f503ddbb-9da8-4a48-8ae4-7f49da3525f2 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:30.977770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f503ddbb-9da8-4a48-8ae4-7f49da3525f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:31.050905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 486/991] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:31.326606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdce6fb-8214-4a3e-b5f0-93168f19d449 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:31.433647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdce6fb-8214-4a3e-b5f0-93168f19d449 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:31.468258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5470b-fcef-4b61-8a04-b992d4072b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:31.468911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5470b-fcef-4b61-8a04-b992d4072b56 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:31.469190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5470b-fcef-4b61-8a04-b992d4072b56 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:31.469463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5470b-fcef-4b61-8a04-b992d4072b56 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:31.470069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5470b-fcef-4b61-8a04-b992d4072b56 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:31.471506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5470b-fcef-4b61-8a04-b992d4072b56 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-69c5470b-fcef-4b61-8a04-b992d4072b56 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:31.482744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:31.483824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:38:31.484670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:31.486096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 506/992] 10.222.0.46 () {56 vars in 1126 bytes} [Fri Sep 9 12:38:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:31.490029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503ddbb-9da8-4a48-8ae4-7f49da3525f2 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:31.500089 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:31.593234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd79081-b870-4f42-ada9-a89da1bcc2ce None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:31.593432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd79081-b870-4f42-ada9-a89da1bcc2ce None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:31.593687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd79081-b870-4f42-ada9-a89da1bcc2ce None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:31.595174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 507/993] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:38:31.599523 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503ddbb-9da8-4a48-8ae4-7f49da3525f2 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:31.603869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abb589-e9e3-4eed-be3b-725bf1cbc003 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:31.604151 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abb589-e9e3-4eed-be3b-725bf1cbc003 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:31.604429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abb589-e9e3-4eed-be3b-725bf1cbc003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:31.625450 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-f1abb589-e9e3-4eed-be3b-725bf1cbc003 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:31.626671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f1abb589-e9e3-4eed-be3b-725bf1cbc003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:31.637743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32cab7-3a32-4dfc-bed1-dece197905e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:31.638870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32cab7-3a32-4dfc-bed1-dece197905e8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:31.639207 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32cab7-3a32-4dfc-bed1-dece197905e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:31.639801 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32cab7-3a32-4dfc-bed1-dece197905e8 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:31.640433 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32cab7-3a32-4dfc-bed1-dece197905e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:31.640735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32cab7-3a32-4dfc-bed1-dece197905e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:31.642102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32cab7-3a32-4dfc-bed1-dece197905e8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:31.653369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 487/994] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 12:38:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:31.655325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32cab7-3a32-4dfc-bed1-dece197905e8 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:31.674912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:31.699600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 508/995] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:31.734063 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:31.764604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32cab7-3a32-4dfc-bed1-dece197905e8 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:31.798869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb8fde-3423-49de-a23b-ced69a1225ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:31.799474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8fde-3423-49de-a23b-ced69a1225ae None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:31.799757 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8fde-3423-49de-a23b-ced69a1225ae None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:31.800029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8fde-3423-49de-a23b-ced69a1225ae None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:31.800647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb8fde-3423-49de-a23b-ced69a1225ae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:31.800948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb8fde-3423-49de-a23b-ced69a1225ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:31.802298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb8fde-3423-49de-a23b-ced69a1225ae None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:31.812645 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 488/996] 10.222.0.46 () {56 vars in 1122 bytes} [Fri Sep 9 12:38:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:31.814264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abb589-e9e3-4eed-be3b-725bf1cbc003 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:31.918131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abb589-e9e3-4eed-be3b-725bf1cbc003 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:31.953319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337f06d-90f9-46f7-a072-1e0a5c6c4c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:31.953929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337f06d-90f9-46f7-a072-1e0a5c6c4c47 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:31.954187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337f06d-90f9-46f7-a072-1e0a5c6c4c47 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:31.954462 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337f06d-90f9-46f7-a072-1e0a5c6c4c47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:31.955174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337f06d-90f9-46f7-a072-1e0a5c6c4c47 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:31.956519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337f06d-90f9-46f7-a072-1e0a5c6c4c47 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:31.967791 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb8fde-3423-49de-a23b-ced69a1225ae None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:32.034613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 509/997] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:38:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:38:32.088634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb8fde-3423-49de-a23b-ced69a1225ae None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:32.134941 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:32.141378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406866e-f130-480a-9a58-6aba047c67be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:32.142019 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406866e-f130-480a-9a58-6aba047c67be None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:32.142295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406866e-f130-480a-9a58-6aba047c67be None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:32.142591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406866e-f130-480a-9a58-6aba047c67be None admin] PATH_INFO: `/v3/services/compute` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:32.143169 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406866e-f130-480a-9a58-6aba047c67be None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:32.144590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406866e-f130-480a-9a58-6aba047c67be None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-6406866e-f130-480a-9a58-6aba047c67be None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:32.154102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:32.155407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:38:32.156597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:32.157355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcaba9-016b-4118-9120-9cd94959987e None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:32.157355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcaba9-016b-4118-9120-9cd94959987e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:32.157355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcaba9-016b-4118-9120-9cd94959987e None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:32.158176 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 510/998] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:38:32.158469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 489/999] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:32.160835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcaba9-016b-4118-9120-9cd94959987e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:32.163742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31591f2b-7420-42ed-9778-f54705eb09e7 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:32.164033 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31591f2b-7420-42ed-9778-f54705eb09e7 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:32.164631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31591f2b-7420-42ed-9778-f54705eb09e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:32.185446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-31591f2b-7420-42ed-9778-f54705eb09e7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:32.187547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-31591f2b-7420-42ed-9778-f54705eb09e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:32.208725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:32.269696 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 490/1000] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:32 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:32.287908 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcaba9-016b-4118-9120-9cd94959987e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:32.322767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46428438-213e-4c3a-8de5-030faaa3555d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:32.323370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46428438-213e-4c3a-8de5-030faaa3555d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:32.323644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46428438-213e-4c3a-8de5-030faaa3555d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:32.323911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46428438-213e-4c3a-8de5-030faaa3555d None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:32.324525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46428438-213e-4c3a-8de5-030faaa3555d None admin] RBAC: Adding query filter params (name=compute) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:32.324819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46428438-213e-4c3a-8de5-030faaa3555d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:32.326156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46428438-213e-4c3a-8de5-030faaa3555d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:32.331322 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 511/1001] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 12:38:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:32.339826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31591f2b-7420-42ed-9778-f54705eb09e7 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:32.447414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31591f2b-7420-42ed-9778-f54705eb09e7 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:32.470591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:32.481425 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdac66-4d95-4349-bdf5-fff3da716fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:32.483902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdac66-4d95-4349-bdf5-fff3da716fb1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:32.483902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdac66-4d95-4349-bdf5-fff3da716fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:32.483902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdac66-4d95-4349-bdf5-fff3da716fb1 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:32.483902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdac66-4d95-4349-bdf5-fff3da716fb1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:32.483902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdac66-4d95-4349-bdf5-fff3da716fb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:32.484791 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdac66-4d95-4349-bdf5-fff3da716fb1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:32.494514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 491/1002] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 12:38:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:32.496293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46428438-213e-4c3a-8de5-030faaa3555d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:32.548821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdac66-4d95-4349-bdf5-fff3da716fb1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:32.598638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:32.601575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46428438-213e-4c3a-8de5-030faaa3555d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:32.637511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e25e1-8b9d-427b-b46f-8780fccb6068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:32.638122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e25e1-8b9d-427b-b46f-8780fccb6068 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:32.638403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e25e1-8b9d-427b-b46f-8780fccb6068 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:32.638703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e25e1-8b9d-427b-b46f-8780fccb6068 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:32.639406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e25e1-8b9d-427b-b46f-8780fccb6068 None admin] RBAC: Adding query filter params (interface=public, service_id=4608f0aa98c64c519b916d217f4f3de8, region_id=RegionOne) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:32.639712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e25e1-8b9d-427b-b46f-8780fccb6068 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:32.641050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e25e1-8b9d-427b-b46f-8780fccb6068 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:32.646437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 512/1003] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 12:38:32 2022] GET /identity/v3/endpoints?service_id=4608f0aa98c64c519b916d217f4f3de8&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:32.652083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e25e1-8b9d-427b-b46f-8780fccb6068 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:32.669648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdac66-4d95-4349-bdf5-fff3da716fb1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:32.710224 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b89ce-b536-4eae-946d-7c0b181e381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:32.710832 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b89ce-b536-4eae-946d-7c0b181e381f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:32.711118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b89ce-b536-4eae-946d-7c0b181e381f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:32.711385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b89ce-b536-4eae-946d-7c0b181e381f None admin] PATH_INFO: `/v3/services/image` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:32.711956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b89ce-b536-4eae-946d-7c0b181e381f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:32.713315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b89ce-b536-4eae-946d-7c0b181e381f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-058b89ce-b536-4eae-946d-7c0b181e381f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:32.722097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:32.723348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:32.724686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 492/1004] 10.222.0.46 () {56 vars in 1105 bytes} [Fri Sep 9 12:38:32 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:32.729036 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b89ce-b536-4eae-946d-7c0b181e381f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:32.760179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e25e1-8b9d-427b-b46f-8780fccb6068 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:32.786414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:32.797543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa134f2-c064-4556-b084-2f73c5f3ecfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:32.798153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa134f2-c064-4556-b084-2f73c5f3ecfc None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:32.798443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa134f2-c064-4556-b084-2f73c5f3ecfc None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:32.798754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa134f2-c064-4556-b084-2f73c5f3ecfc None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:32.799364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa134f2-c064-4556-b084-2f73c5f3ecfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:32.800713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa134f2-c064-4556-b084-2f73c5f3ecfc None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:32.813184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 513/1005] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 12:38:32 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:38:32.843491 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b89ce-b536-4eae-946d-7c0b181e381f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:32.882292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc243ada-f43f-4f3d-9e30-b2b7279353f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:32.882944 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243ada-f43f-4f3d-9e30-b2b7279353f2 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:32.883221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243ada-f43f-4f3d-9e30-b2b7279353f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:32.883488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243ada-f43f-4f3d-9e30-b2b7279353f2 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:32.884114 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc243ada-f43f-4f3d-9e30-b2b7279353f2 None admin] RBAC: Adding query filter params (name=image) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:32.884412 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc243ada-f43f-4f3d-9e30-b2b7279353f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:32.885884 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc243ada-f43f-4f3d-9e30-b2b7279353f2 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:32.891673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 493/1006] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:32.897719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa134f2-c064-4556-b084-2f73c5f3ecfc None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:32.927553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:33.009237 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa134f2-c064-4556-b084-2f73c5f3ecfc None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:33.043526 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1ad8c-eef5-4412-b4f8-73dcced0bdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:33.044125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1ad8c-eef5-4412-b4f8-73dcced0bdc7 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:33.044401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1ad8c-eef5-4412-b4f8-73dcced0bdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:33.044668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1ad8c-eef5-4412-b4f8-73dcced0bdc7 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:33.045312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1ad8c-eef5-4412-b4f8-73dcced0bdc7 None admin] RBAC: Adding query filter params (type=image) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:33.045610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1ad8c-eef5-4412-b4f8-73dcced0bdc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:33.046989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1ad8c-eef5-4412-b4f8-73dcced0bdc7 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:33.051524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 514/1007] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:32 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:34.236436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac4371-7cd5-4ecd-8679-45bea3dc2d58 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:34.236970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac4371-7cd5-4ecd-8679-45bea3dc2d58 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:34.236970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac4371-7cd5-4ecd-8679-45bea3dc2d58 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:34.238339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 494/1008] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:34.244419 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1adb8-3a07-452d-bf80-fecb36c4c1a2 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:34.244822 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1adb8-3a07-452d-bf80-fecb36c4c1a2 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:34.245007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1adb8-3a07-452d-bf80-fecb36c4c1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:34.263461 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-a2a1adb8-3a07-452d-bf80-fecb36c4c1a2 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:34.264644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a2a1adb8-3a07-452d-bf80-fecb36c4c1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:34.332275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 515/1009] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:34.600870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac4371-7cd5-4ecd-8679-45bea3dc2d58 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:34.705016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac4371-7cd5-4ecd-8679-45bea3dc2d58 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:34.721329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:34.739614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b173d77-77cb-44a8-b06f-3a9c3c3912a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:34.740234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b173d77-77cb-44a8-b06f-3a9c3c3912a1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:34.740512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b173d77-77cb-44a8-b06f-3a9c3c3912a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:34.741353 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b173d77-77cb-44a8-b06f-3a9c3c3912a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:34.742468 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b173d77-77cb-44a8-b06f-3a9c3c3912a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:34.743912 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b173d77-77cb-44a8-b06f-3a9c3c3912a1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:34.745619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 495/1010] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:34.751406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1adb8-3a07-452d-bf80-fecb36c4c1a2 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:34.776432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:34.853031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1adb8-3a07-452d-bf80-fecb36c4c1a2 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:34.888248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673d84e-7ea7-414d-a2e7-4349e103ed96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:34.888866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673d84e-7ea7-414d-a2e7-4349e103ed96 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:34.889132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673d84e-7ea7-414d-a2e7-4349e103ed96 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:34.889402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673d84e-7ea7-414d-a2e7-4349e103ed96 None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:34.890059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673d84e-7ea7-414d-a2e7-4349e103ed96 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:34.891493 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673d84e-7ea7-414d-a2e7-4349e103ed96 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:34.970749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 516/1011] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:38:34 2022] POST /identity/v3/projects => generated 311 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:38:35.068756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19240a83-1208-4766-a372-7f438fd09b26 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:35.069065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19240a83-1208-4766-a372-7f438fd09b26 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:35.069414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19240a83-1208-4766-a372-7f438fd09b26 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:35.071078 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 496/1012] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:35.077025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6750ee-2691-4682-9648-3575ea019dfa None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:35.077411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6750ee-2691-4682-9648-3575ea019dfa None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:35.077575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6750ee-2691-4682-9648-3575ea019dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:35.098341 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-2e6750ee-2691-4682-9648-3575ea019dfa None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:35.102661 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2e6750ee-2691-4682-9648-3575ea019dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:35.181978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 517/1013] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:35 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:35.201268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ca99d-70a2-40f6-9bf6-76653fb72c6b None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:35.201555 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ca99d-70a2-40f6-9bf6-76653fb72c6b None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:35.201830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ca99d-70a2-40f6-9bf6-76653fb72c6b None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:35.203330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 497/1014] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:35.208339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1aa32-055f-4cb7-8d1e-44ac7fc4a161 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:35.208632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1aa32-055f-4cb7-8d1e-44ac7fc4a161 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:35.208910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1aa32-055f-4cb7-8d1e-44ac7fc4a161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:35.217606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:35.228084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d1d1aa32-055f-4cb7-8d1e-44ac7fc4a161 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:35.233677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d1d1aa32-055f-4cb7-8d1e-44ac7fc4a161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:35.298194 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 518/1015] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:35 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:35.472167 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ca99d-70a2-40f6-9bf6-76653fb72c6b None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:35.577013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1aa32-055f-4cb7-8d1e-44ac7fc4a161 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:35.585836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ca99d-70a2-40f6-9bf6-76653fb72c6b None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:35.620549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a07f8-f0eb-4e35-aeca-d39f6999b758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:35.621185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a07f8-f0eb-4e35-aeca-d39f6999b758 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:35.621468 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a07f8-f0eb-4e35-aeca-d39f6999b758 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:35.621737 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a07f8-f0eb-4e35-aeca-d39f6999b758 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:35.622316 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a07f8-f0eb-4e35-aeca-d39f6999b758 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:35.623709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a07f8-f0eb-4e35-aeca-d39f6999b758 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-750a07f8-f0eb-4e35-aeca-d39f6999b758 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:35.632934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:35.634012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:38:35.634903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:35.638904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 498/1016] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:35.640229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a07f8-f0eb-4e35-aeca-d39f6999b758 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:35.688362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1aa32-055f-4cb7-8d1e-44ac7fc4a161 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:35.692719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:35.723103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c5991-b631-4621-a73b-b40081b3d966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:35.723631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c5991-b631-4621-a73b-b40081b3d966 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:35.723905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c5991-b631-4621-a73b-b40081b3d966 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:35.724223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c5991-b631-4621-a73b-b40081b3d966 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:35.725001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c5991-b631-4621-a73b-b40081b3d966 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:35.726806 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c5991-b631-4621-a73b-b40081b3d966 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:35.730265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:35.749182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a07f8-f0eb-4e35-aeca-d39f6999b758 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:35.783118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94c299-273c-4958-bc57-87c6efe21ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:35.783726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94c299-273c-4958-bc57-87c6efe21ad4 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:35.784004 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94c299-273c-4958-bc57-87c6efe21ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:35.784273 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94c299-273c-4958-bc57-87c6efe21ad4 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:35.784889 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d94c299-273c-4958-bc57-87c6efe21ad4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:35.785186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d94c299-273c-4958-bc57-87c6efe21ad4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:35.786563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d94c299-273c-4958-bc57-87c6efe21ad4 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:35.791298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 499/1017] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 12:38:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:35.796841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94c299-273c-4958-bc57-87c6efe21ad4 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:35.829185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 519/1018] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:38:35 2022] POST /identity/v3/services => generated 242 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:38:35.910271 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94c299-273c-4958-bc57-87c6efe21ad4 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:35.927257 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:35.958047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac0286-720e-4c46-ba31-b9afd22b52dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:35.959026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac0286-720e-4c46-ba31-b9afd22b52dc None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:35.959359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac0286-720e-4c46-ba31-b9afd22b52dc None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:35.959641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac0286-720e-4c46-ba31-b9afd22b52dc None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:35.960680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac0286-720e-4c46-ba31-b9afd22b52dc None admin] RBAC: Adding query filter params (type=compute) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:35.961507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac0286-720e-4c46-ba31-b9afd22b52dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:35.962952 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac0286-720e-4c46-ba31-b9afd22b52dc None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:35.972856 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 500/1019] 10.222.0.46 () {56 vars in 1112 bytes} [Fri Sep 9 12:38:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:35.974467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c5991-b631-4621-a73b-b40081b3d966 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:36.080694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c5991-b631-4621-a73b-b40081b3d966 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:36.123710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf81f5-d804-46c7-82ae-714cf1f6f95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:36.124344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf81f5-d804-46c7-82ae-714cf1f6f95d None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:36.124624 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf81f5-d804-46c7-82ae-714cf1f6f95d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:36.124896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf81f5-d804-46c7-82ae-714cf1f6f95d None admin] PATH_INFO: `/v3/endpoints` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:36.125606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf81f5-d804-46c7-82ae-714cf1f6f95d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:36.127076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf81f5-d804-46c7-82ae-714cf1f6f95d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:36.206865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 520/1020] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:38:35 2022] POST /identity/v3/endpoints => generated 334 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:38:37.230105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754bb2c-337b-4955-841f-95f6fe5e481a None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:37.230637 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754bb2c-337b-4955-841f-95f6fe5e481a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:37.230714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754bb2c-337b-4955-841f-95f6fe5e481a None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:37.232163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 501/1021] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:37.237203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f9252-527c-4cd0-8e76-bd5a75623c4c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:37.237572 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f9252-527c-4cd0-8e76-bd5a75623c4c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:37.237752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f9252-527c-4cd0-8e76-bd5a75623c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:37.257546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d85f9252-527c-4cd0-8e76-bd5a75623c4c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:37.258760 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d85f9252-527c-4cd0-8e76-bd5a75623c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:37.304379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:37.335716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 521/1022] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:37.601415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754bb2c-337b-4955-841f-95f6fe5e481a None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:37.702224 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754bb2c-337b-4955-841f-95f6fe5e481a None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:37.735962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c5c4e-8fb1-4b8b-9bd6-84aa3e5ff8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:37.736585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c5c4e-8fb1-4b8b-9bd6-84aa3e5ff8fe None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:37.736870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c5c4e-8fb1-4b8b-9bd6-84aa3e5ff8fe None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:37.737147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c5c4e-8fb1-4b8b-9bd6-84aa3e5ff8fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:37.738264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c5c4e-8fb1-4b8b-9bd6-84aa3e5ff8fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:37.739726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c5c4e-8fb1-4b8b-9bd6-84aa3e5ff8fe None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:37.742182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 502/1023] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:37.747656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f9252-527c-4cd0-8e76-bd5a75623c4c None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:37.844560 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f9252-527c-4cd0-8e76-bd5a75623c4c None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:37.878904 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c3635-6005-4384-a7bf-f19de02cbc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:37.879428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c3635-6005-4384-a7bf-f19de02cbc43 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:37.879704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c3635-6005-4384-a7bf-f19de02cbc43 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:37.879978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c3635-6005-4384-a7bf-f19de02cbc43 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:37.880626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c3635-6005-4384-a7bf-f19de02cbc43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:37.882696 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c3635-6005-4384-a7bf-f19de02cbc43 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:37.971572 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 522/1024] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:38:37 2022] POST /identity/v3/users => generated 285 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:38:38.015795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071f9b7-70fe-4b6e-8294-c0a9fa8a3287 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:38.016162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071f9b7-70fe-4b6e-8294-c0a9fa8a3287 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:38.016513 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071f9b7-70fe-4b6e-8294-c0a9fa8a3287 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:38.018077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 503/1025] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:38:38.024113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bb7df-e981-4b53-b979-3eec728d27b9 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:38.024470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bb7df-e981-4b53-b979-3eec728d27b9 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:38.024666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bb7df-e981-4b53-b979-3eec728d27b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:38.044240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-fc7bb7df-e981-4b53-b979-3eec728d27b9 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:38.045675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fc7bb7df-e981-4b53-b979-3eec728d27b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:38.077038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:38.129921 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 523/1026] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:38.399771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071f9b7-70fe-4b6e-8294-c0a9fa8a3287 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:38.445175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:38.503509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60d8ff-c00b-4500-a6a9-f58a4835da84 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:38.504055 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60d8ff-c00b-4500-a6a9-f58a4835da84 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:38.504055 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60d8ff-c00b-4500-a6a9-f58a4835da84 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:38.504515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071f9b7-70fe-4b6e-8294-c0a9fa8a3287 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:38.505381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 524/1027] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:38.511113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac1af5-9ae6-440c-82cd-96b1bc83ee89 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:38.511389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac1af5-9ae6-440c-82cd-96b1bc83ee89 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:38.511661 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac1af5-9ae6-440c-82cd-96b1bc83ee89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:38.530864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-9aac1af5-9ae6-440c-82cd-96b1bc83ee89 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:38.532245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9aac1af5-9ae6-440c-82cd-96b1bc83ee89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:38.545907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414ddac-3982-4d13-a1ed-2a0a0f99201f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:38.546567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414ddac-3982-4d13-a1ed-2a0a0f99201f None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:38.546860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414ddac-3982-4d13-a1ed-2a0a0f99201f None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:38.547131 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414ddac-3982-4d13-a1ed-2a0a0f99201f None admin] PATH_INFO: `/v3/services/image` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:38.547704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414ddac-3982-4d13-a1ed-2a0a0f99201f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:38.549127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414ddac-3982-4d13-a1ed-2a0a0f99201f None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-b414ddac-3982-4d13-a1ed-2a0a0f99201f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:38.558005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:38.559158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:38.560751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 504/1028] 10.222.0.46 () {56 vars in 1105 bytes} [Fri Sep 9 12:38:38 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:38.565136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414ddac-3982-4d13-a1ed-2a0a0f99201f None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:38.610099 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 525/1029] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:38.672095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414ddac-3982-4d13-a1ed-2a0a0f99201f None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:38.675669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:38.709901 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3af79-8b4d-441e-a3fd-20dd68210387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:38.710706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3af79-8b4d-441e-a3fd-20dd68210387 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:38.710988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3af79-8b4d-441e-a3fd-20dd68210387 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:38.711259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3af79-8b4d-441e-a3fd-20dd68210387 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:38.711883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3af79-8b4d-441e-a3fd-20dd68210387 None admin] RBAC: Adding query filter params (name=image) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:38.712233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3af79-8b4d-441e-a3fd-20dd68210387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:38.713662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3af79-8b4d-441e-a3fd-20dd68210387 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:38.723174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 505/1030] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:38.724515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac1af5-9ae6-440c-82cd-96b1bc83ee89 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:38.817381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:38.826847 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac1af5-9ae6-440c-82cd-96b1bc83ee89 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:38.858955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52382f1-929f-479f-be02-f1391b3116e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:38.859551 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52382f1-929f-479f-be02-f1391b3116e6 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:38.859826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52382f1-929f-479f-be02-f1391b3116e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:38.860093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52382f1-929f-479f-be02-f1391b3116e6 None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:38.860760 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52382f1-929f-479f-be02-f1391b3116e6 None admin] RBAC: Adding query filter params (type=image) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:38.861317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52382f1-929f-479f-be02-f1391b3116e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:38.862966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52382f1-929f-479f-be02-f1391b3116e6 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:38.873402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 526/1031] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:38.874348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3af79-8b4d-441e-a3fd-20dd68210387 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:38.883144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52382f1-929f-479f-be02-f1391b3116e6 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:38.981174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3af79-8b4d-441e-a3fd-20dd68210387 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:38.993961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52382f1-929f-479f-be02-f1391b3116e6 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:39.032476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0334dd-4328-4f41-9bf3-3f8731a1b99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:39.033182 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0334dd-4328-4f41-9bf3-3f8731a1b99c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:39.033467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0334dd-4328-4f41-9bf3-3f8731a1b99c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:39.033741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0334dd-4328-4f41-9bf3-3f8731a1b99c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:39.035218 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90fe14-f251-4fe0-9d02-4ffe90f92d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:39.036059 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90fe14-f251-4fe0-9d02-4ffe90f92d80 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:39.036344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90fe14-f251-4fe0-9d02-4ffe90f92d80 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:39.036623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90fe14-f251-4fe0-9d02-4ffe90f92d80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:39.037911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90fe14-f251-4fe0-9d02-4ffe90f92d80 None admin] RBAC: Adding query filter params (interface=public, service_id=ce0f7de8f496461c8b50ea0514f14c79, region_id=RegionOne) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:39.038442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90fe14-f251-4fe0-9d02-4ffe90f92d80 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:39.040117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90fe14-f251-4fe0-9d02-4ffe90f92d80 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:39.052734 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 506/1032] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 12:38:38 2022] GET /identity/v3/endpoints?service_id=ce0f7de8f496461c8b50ea0514f14c79&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:39.054661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90fe14-f251-4fe0-9d02-4ffe90f92d80 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:39.055206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0334dd-4328-4f41-9bf3-3f8731a1b99c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-bf0334dd-4328-4f41-9bf3-3f8731a1b99c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:39.060559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:39.061769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:39.062658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:39.062658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:39.062658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:39.062658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:39.062658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:39.062658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:39.062658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:39.062658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:39.062658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:39.062658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:39.062658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:38:39.062658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:39.066924 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 527/1033] 10.222.0.46 () {56 vars in 1120 bytes} [Fri Sep 9 12:38:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:39.068152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0334dd-4328-4f41-9bf3-3f8731a1b99c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:39.088477 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:39.092848 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:39.164997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90fe14-f251-4fe0-9d02-4ffe90f92d80 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:39.183296 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0334dd-4328-4f41-9bf3-3f8731a1b99c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:39.202993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d3ff1-1f96-4069-8421-3599168faf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:39.203598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d3ff1-1f96-4069-8421-3599168faf82 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:39.203877 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d3ff1-1f96-4069-8421-3599168faf82 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:39.204147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d3ff1-1f96-4069-8421-3599168faf82 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:39.204740 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d3ff1-1f96-4069-8421-3599168faf82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:39.206093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d3ff1-1f96-4069-8421-3599168faf82 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:39.213751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 507/1034] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 12:38:39 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:38:39.220724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da1752-af33-422b-b3fa-e9c4651d4b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:39.221345 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da1752-af33-422b-b3fa-e9c4651d4b95 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:39.221619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da1752-af33-422b-b3fa-e9c4651d4b95 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:39.221882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da1752-af33-422b-b3fa-e9c4651d4b95 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:39.222596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da1752-af33-422b-b3fa-e9c4651d4b95 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:39.222897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da1752-af33-422b-b3fa-e9c4651d4b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:39.224335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da1752-af33-422b-b3fa-e9c4651d4b95 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:39.234057 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 528/1035] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 12:38:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:39.239847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d3ff1-1f96-4069-8421-3599168faf82 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:39.351798 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:39.358824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d3ff1-1f96-4069-8421-3599168faf82 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:39.398260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9e46d-fa2b-4080-994d-46bc503a1bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:39.398962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9e46d-fa2b-4080-994d-46bc503a1bef None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:39.399242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9e46d-fa2b-4080-994d-46bc503a1bef None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:39.399519 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9e46d-fa2b-4080-994d-46bc503a1bef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:39.411657 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9e46d-fa2b-4080-994d-46bc503a1bef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:39.414335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9e46d-fa2b-4080-994d-46bc503a1bef None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-91b9e46d-fa2b-4080-994d-46bc503a1bef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:39.429777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:39.430874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:39.431795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:39.431795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:39.431795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:39.431795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 508/1036] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:39.437148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da1752-af33-422b-b3fa-e9c4651d4b95 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:39.516047 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:39.537599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da1752-af33-422b-b3fa-e9c4651d4b95 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:39.571264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2011daa-4ab2-433e-a7fa-88bad2069dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:39.571871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2011daa-4ab2-433e-a7fa-88bad2069dcf None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:39.572145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2011daa-4ab2-433e-a7fa-88bad2069dcf None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:39.572412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2011daa-4ab2-433e-a7fa-88bad2069dcf None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:39.573051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2011daa-4ab2-433e-a7fa-88bad2069dcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:39.573344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2011daa-4ab2-433e-a7fa-88bad2069dcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:39.574704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2011daa-4ab2-433e-a7fa-88bad2069dcf None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:39.590916 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 529/1037] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:39.592334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9e46d-fa2b-4080-994d-46bc503a1bef None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:39.698946 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9e46d-fa2b-4080-994d-46bc503a1bef None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:39.744186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f215c1-66bf-4268-b816-f9993603fe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:39.744823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f215c1-66bf-4268-b816-f9993603fe6c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:39.745109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f215c1-66bf-4268-b816-f9993603fe6c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:39.745379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f215c1-66bf-4268-b816-f9993603fe6c None admin] PATH_INFO: `/v3/users/nova` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:39.753097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f215c1-66bf-4268-b816-f9993603fe6c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:39.754547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f215c1-66bf-4268-b816-f9993603fe6c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-d1f215c1-66bf-4268-b816-f9993603fe6c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:39.765493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:39.766793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:39.767940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:39.767940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:39.767940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:39.767940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:39.767940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:39.767940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:39.767940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:39.767940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:39.767940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:38:39.767940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:39.770814 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 509/1038] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 12:38:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:39.772349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2011daa-4ab2-433e-a7fa-88bad2069dcf None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:39.871242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2011daa-4ab2-433e-a7fa-88bad2069dcf None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:39.903791 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36a11c-83de-459e-a640-e7a6699337bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:39.904393 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36a11c-83de-459e-a640-e7a6699337bf None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:39.904668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36a11c-83de-459e-a640-e7a6699337bf None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:39.904939 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36a11c-83de-459e-a640-e7a6699337bf None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:39.905655 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36a11c-83de-459e-a640-e7a6699337bf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:39.905952 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36a11c-83de-459e-a640-e7a6699337bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:39.907407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36a11c-83de-459e-a640-e7a6699337bf None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:39.915682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:39.925844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 530/1039] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 12:38:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:39.931728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f215c1-66bf-4268-b816-f9993603fe6c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:39.943506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:40.046128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f215c1-66bf-4268-b816-f9993603fe6c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:40.080590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4daeb8-fb39-48e0-a649-d42a950a57f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:40.081196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4daeb8-fb39-48e0-a649-d42a950a57f6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:40.081476 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4daeb8-fb39-48e0-a649-d42a950a57f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:40.081750 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4daeb8-fb39-48e0-a649-d42a950a57f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:40.166285 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4daeb8-fb39-48e0-a649-d42a950a57f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:40.167780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4daeb8-fb39-48e0-a649-d42a950a57f6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:40.178989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:40.215087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32bd89-9690-47b1-8056-42991dce876d None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:40.215427 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32bd89-9690-47b1-8056-42991dce876d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:40.215625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32bd89-9690-47b1-8056-42991dce876d None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:40.217192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 531/1040] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:38:40.222065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810374f3-832c-4212-9341-132d50e87763 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:40.222597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810374f3-832c-4212-9341-132d50e87763 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:40.223080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810374f3-832c-4212-9341-132d50e87763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:40.243111 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-810374f3-832c-4212-9341-132d50e87763 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:40.244363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-810374f3-832c-4212-9341-132d50e87763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:40.277354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 510/1041] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:38:39 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:38:40.282989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4daeb8-fb39-48e0-a649-d42a950a57f6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:40.313196 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 532/1042] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:40.389253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4daeb8-fb39-48e0-a649-d42a950a57f6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:40.398443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:40.422760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822f67d-0fbe-490f-82cf-ddbd20dbb202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:40.423391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822f67d-0fbe-490f-82cf-ddbd20dbb202 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:40.423644 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822f67d-0fbe-490f-82cf-ddbd20dbb202 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:40.423911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822f67d-0fbe-490f-82cf-ddbd20dbb202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:40.435192 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822f67d-0fbe-490f-82cf-ddbd20dbb202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:40.436864 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822f67d-0fbe-490f-82cf-ddbd20dbb202 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-a822f67d-0fbe-490f-82cf-ddbd20dbb202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:40.451974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:40.452933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:40.458714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:40.458714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:40.458714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:40.458714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 511/1043] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:40.458946 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810374f3-832c-4212-9341-132d50e87763 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:40.556930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810374f3-832c-4212-9341-132d50e87763 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:40.583159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:40.584355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822f67d-0fbe-490f-82cf-ddbd20dbb202 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:40.592637 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdfd7f-133f-49a7-83cb-2efac1125480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:40.593277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdfd7f-133f-49a7-83cb-2efac1125480 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:40.593525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdfd7f-133f-49a7-83cb-2efac1125480 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:40.595999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdfd7f-133f-49a7-83cb-2efac1125480 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:40.595999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdfd7f-133f-49a7-83cb-2efac1125480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:40.595999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdfd7f-133f-49a7-83cb-2efac1125480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:40.596239 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdfd7f-133f-49a7-83cb-2efac1125480 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:40.608546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 533/1044] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:40.613999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdfd7f-133f-49a7-83cb-2efac1125480 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:40.692778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822f67d-0fbe-490f-82cf-ddbd20dbb202 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:40.716598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdfd7f-133f-49a7-83cb-2efac1125480 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:40.728765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f98a3-e407-47fa-aaa7-deec829e0dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:40.729551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f98a3-e407-47fa-aaa7-deec829e0dd8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:40.729842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f98a3-e407-47fa-aaa7-deec829e0dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:40.730119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f98a3-e407-47fa-aaa7-deec829e0dd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:40.741613 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f98a3-e407-47fa-aaa7-deec829e0dd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-351f98a3-e407-47fa-aaa7-deec829e0dd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:40.746395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:40.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:40.748345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:40.748345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:40.748345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:40.748345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:40.748345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:40.748345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:40.748345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:40.748345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:40.748345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:40.748345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:40.748345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:38:40.748345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:40.748345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 512/1045] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:38:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:40.753227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f98a3-e407-47fa-aaa7-deec829e0dd8 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:40.754417 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641fad8-3282-4fde-b2cb-dc4706e907de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:40.755068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641fad8-3282-4fde-b2cb-dc4706e907de None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:40.755345 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641fad8-3282-4fde-b2cb-dc4706e907de None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:40.755605 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641fad8-3282-4fde-b2cb-dc4706e907de None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:40.762969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641fad8-3282-4fde-b2cb-dc4706e907de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:40.764401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641fad8-3282-4fde-b2cb-dc4706e907de None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-d641fad8-3282-4fde-b2cb-dc4706e907de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:40.776888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:40.777799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:40.778726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:40.778726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:40.778726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:40.778726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:40.778726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:40.778726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:40.778726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:40.778726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:40.779193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:40.779267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 534/1046] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:40.783870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641fad8-3282-4fde-b2cb-dc4706e907de None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:40.833805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:40.862153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f98a3-e407-47fa-aaa7-deec829e0dd8 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:40.891528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641fad8-3282-4fde-b2cb-dc4706e907de None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:40.897757 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859be55-53b4-468f-8484-ba34bf6e32eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:40.898398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859be55-53b4-468f-8484-ba34bf6e32eb None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:40.898738 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859be55-53b4-468f-8484-ba34bf6e32eb None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:40.899426 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859be55-53b4-468f-8484-ba34bf6e32eb None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:40.900096 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7859be55-53b4-468f-8484-ba34bf6e32eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:40.900402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7859be55-53b4-468f-8484-ba34bf6e32eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:40.902123 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7859be55-53b4-468f-8484-ba34bf6e32eb None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:40.911561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 513/1047] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:38:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:40.917729 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859be55-53b4-468f-8484-ba34bf6e32eb None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:40.926813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1351b-2349-4c7d-adec-5923854146ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:40.927476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1351b-2349-4c7d-adec-5923854146ff None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:40.927775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1351b-2349-4c7d-adec-5923854146ff None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:40.928094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1351b-2349-4c7d-adec-5923854146ff None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:40.928785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1351b-2349-4c7d-adec-5923854146ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:40.929080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1351b-2349-4c7d-adec-5923854146ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:40.930511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1351b-2349-4c7d-adec-5923854146ff None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:40.943137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 535/1048] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:40.949152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1351b-2349-4c7d-adec-5923854146ff None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:41.005516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:41.028076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859be55-53b4-468f-8484-ba34bf6e32eb None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:41.049863 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1351b-2349-4c7d-adec-5923854146ff None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:41.061350 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:41.065499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823d8d9-806b-4572-8ed0-aa783eff7f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:41.066141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823d8d9-806b-4572-8ed0-aa783eff7f50 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:41.066419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823d8d9-806b-4572-8ed0-aa783eff7f50 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:41.066766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823d8d9-806b-4572-8ed0-aa783eff7f50 None admin] PATH_INFO: `/v3/users/6928aac4953443dfa32b3d40cf7e149f` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:41.082035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823d8d9-806b-4572-8ed0-aa783eff7f50 None admin] RBAC: Authorizing `identity:get_user(user_id=6928aac4953443dfa32b3d40cf7e149f)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:41.083502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823d8d9-806b-4572-8ed0-aa783eff7f50 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:41.086142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 514/1049] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:38:40 2022] GET /identity/v3/users/6928aac4953443dfa32b3d40cf7e149f => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:41.091511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823d8d9-806b-4572-8ed0-aa783eff7f50 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:41.099777 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb40e7-191c-441f-b7c0-9fd6ca8f02bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:41.101265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb40e7-191c-441f-b7c0-9fd6ca8f02bd None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:41.101265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb40e7-191c-441f-b7c0-9fd6ca8f02bd None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:41.101515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb40e7-191c-441f-b7c0-9fd6ca8f02bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:41.102205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb40e7-191c-441f-b7c0-9fd6ca8f02bd None admin] RBAC: Adding query filter params (role.id=0f979abba91c44569a1ceb70bceceacb, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=f29af98c5fba4569b16edf240b13c5c5) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:41.105455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb40e7-191c-441f-b7c0-9fd6ca8f02bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:41.105455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb40e7-191c-441f-b7c0-9fd6ca8f02bd None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:41.111013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 536/1050] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:38:40 2022] GET /identity/v3/role_assignments?user.id=f29af98c5fba4569b16edf240b13c5c5&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=0f979abba91c44569a1ceb70bceceacb => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:41.211303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823d8d9-806b-4572-8ed0-aa783eff7f50 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:41.247123 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbf04e-7ab1-46ec-9c61-dba1761e5cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:41.247738 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbf04e-7ab1-46ec-9c61-dba1761e5cb7 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:41.248017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbf04e-7ab1-46ec-9c61-dba1761e5cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:41.248288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbf04e-7ab1-46ec-9c61-dba1761e5cb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:41.269738 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:41.331302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbf04e-7ab1-46ec-9c61-dba1761e5cb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:41.332778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbf04e-7ab1-46ec-9c61-dba1761e5cb7 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:41.454598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 515/1051] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:38:41 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 12:38:41.455746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb40e7-191c-441f-b7c0-9fd6ca8f02bd None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:41.473805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993b45-f295-4643-bfec-2faca6e105eb None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:41.474352 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993b45-f295-4643-bfec-2faca6e105eb None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:41.474663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993b45-f295-4643-bfec-2faca6e105eb None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:41.477082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 516/1052] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:38:41.483067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79660f-486f-47ef-86ff-d2b147558534 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:41.483344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79660f-486f-47ef-86ff-d2b147558534 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:41.483617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79660f-486f-47ef-86ff-d2b147558534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:41.502178 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-2a79660f-486f-47ef-86ff-d2b147558534 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:41.503373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2a79660f-486f-47ef-86ff-d2b147558534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:41.540930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:41.561542 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb40e7-191c-441f-b7c0-9fd6ca8f02bd None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:41.578533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 517/1053] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:41.603921 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed791ec-da9f-4ab2-b93e-68d680bb2f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:41.604548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed791ec-da9f-4ab2-b93e-68d680bb2f07 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:41.604823 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed791ec-da9f-4ab2-b93e-68d680bb2f07 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:41.605091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed791ec-da9f-4ab2-b93e-68d680bb2f07 None admin] PATH_INFO: `/v3/projects/8bfb7d578bd241fa9841964b4d3410e4` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:41.606189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed791ec-da9f-4ab2-b93e-68d680bb2f07 None admin] RBAC: Authorizing `identity:get_project(project_id=8bfb7d578bd241fa9841964b4d3410e4)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:41.607648 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed791ec-da9f-4ab2-b93e-68d680bb2f07 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:41.609494 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 537/1054] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:38:41 2022] GET /identity/v3/projects/8bfb7d578bd241fa9841964b4d3410e4 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:41.615331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79660f-486f-47ef-86ff-d2b147558534 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:41.718788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79660f-486f-47ef-86ff-d2b147558534 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:41.755005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00e40a-8243-444b-8769-fb6b5cf2bc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:41.755611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00e40a-8243-444b-8769-fb6b5cf2bc3c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:41.755892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00e40a-8243-444b-8769-fb6b5cf2bc3c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:41.756943 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00e40a-8243-444b-8769-fb6b5cf2bc3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:41.758079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00e40a-8243-444b-8769-fb6b5cf2bc3c None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=8bfb7d578bd241fa9841964b4d3410e4, user.id=6928aac4953443dfa32b3d40cf7e149f) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:41.758395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00e40a-8243-444b-8769-fb6b5cf2bc3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:41.759838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00e40a-8243-444b-8769-fb6b5cf2bc3c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:41.765437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 518/1055] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:38:41 2022] GET /identity/v3/role_assignments?user.id=6928aac4953443dfa32b3d40cf7e149f&scope.project.id=8bfb7d578bd241fa9841964b4d3410e4&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:41.856287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed791ec-da9f-4ab2-b93e-68d680bb2f07 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:41.900055 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:41.963289 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed791ec-da9f-4ab2-b93e-68d680bb2f07 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:41.999967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a664f03-24b3-4d8c-a72f-2f0eb2a56534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:42.000517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a664f03-24b3-4d8c-a72f-2f0eb2a56534 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:42.000802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a664f03-24b3-4d8c-a72f-2f0eb2a56534 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:42.001071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a664f03-24b3-4d8c-a72f-2f0eb2a56534 None admin] PATH_INFO: `/v3/services/image` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:42.001655 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a664f03-24b3-4d8c-a72f-2f0eb2a56534 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:42.003036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a664f03-24b3-4d8c-a72f-2f0eb2a56534 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-4a664f03-24b3-4d8c-a72f-2f0eb2a56534 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:42.011914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:42.012933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:38:42.013809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:42.017934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 538/1056] 10.222.0.46 () {56 vars in 1105 bytes} [Fri Sep 9 12:38:41 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:42.018952 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00e40a-8243-444b-8769-fb6b5cf2bc3c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:42.053837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:42.129241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00e40a-8243-444b-8769-fb6b5cf2bc3c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:42.165210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb266a-0c3c-476e-9865-e73c32735e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:42.165824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb266a-0c3c-476e-9865-e73c32735e7a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:42.166108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb266a-0c3c-476e-9865-e73c32735e7a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:42.166382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb266a-0c3c-476e-9865-e73c32735e7a None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:42.167033 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb266a-0c3c-476e-9865-e73c32735e7a None admin] RBAC: Adding query filter params (name=image) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:42.167335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb266a-0c3c-476e-9865-e73c32735e7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:42.168690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb266a-0c3c-476e-9865-e73c32735e7a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:42.178093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 519/1057] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:42.179706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a664f03-24b3-4d8c-a72f-2f0eb2a56534 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:42.285249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a664f03-24b3-4d8c-a72f-2f0eb2a56534 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:42.286724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:42.321253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8891a2-e84d-45c4-988a-a5ca1b9d8b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:42.321865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8891a2-e84d-45c4-988a-a5ca1b9d8b8b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:42.322192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8891a2-e84d-45c4-988a-a5ca1b9d8b8b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:42.322473 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8891a2-e84d-45c4-988a-a5ca1b9d8b8b None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:42.323175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8891a2-e84d-45c4-988a-a5ca1b9d8b8b None admin] RBAC: Adding query filter params (type=image) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:42.323478 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8891a2-e84d-45c4-988a-a5ca1b9d8b8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:42.324832 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8891a2-e84d-45c4-988a-a5ca1b9d8b8b None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:42.329985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 539/1058] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:42.335981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb266a-0c3c-476e-9865-e73c32735e7a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:42.436180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:42.440577 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb266a-0c3c-476e-9865-e73c32735e7a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:42.476653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9f439-462a-4add-933d-99e9708fd440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:42.477278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9f439-462a-4add-933d-99e9708fd440 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:42.477528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9f439-462a-4add-933d-99e9708fd440 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:42.477802 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9f439-462a-4add-933d-99e9708fd440 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:42.478462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9f439-462a-4add-933d-99e9708fd440 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:42.479844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9f439-462a-4add-933d-99e9708fd440 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:42.578599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 520/1059] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:38:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:38:43.247199 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540d95d-2800-4e65-a949-c87ad6b6753a None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:43.247730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540d95d-2800-4e65-a949-c87ad6b6753a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:43.247730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540d95d-2800-4e65-a949-c87ad6b6753a None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:43.249220 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 540/1060] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:43.254048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08895f5-bab1-4bfd-8104-91e07fe4d905 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:43.254336 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08895f5-bab1-4bfd-8104-91e07fe4d905 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:43.254647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08895f5-bab1-4bfd-8104-91e07fe4d905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:43.273609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-a08895f5-bab1-4bfd-8104-91e07fe4d905 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:43.274821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a08895f5-bab1-4bfd-8104-91e07fe4d905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:43.349425 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 521/1061] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:43.625411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540d95d-2800-4e65-a949-c87ad6b6753a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:43.733511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540d95d-2800-4e65-a949-c87ad6b6753a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:43.776512 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c36d62-0bcd-457b-9fcc-dd2250a20eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:43.777135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c36d62-0bcd-457b-9fcc-dd2250a20eb2 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:43.777414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c36d62-0bcd-457b-9fcc-dd2250a20eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:43.777685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c36d62-0bcd-457b-9fcc-dd2250a20eb2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:43.788467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c36d62-0bcd-457b-9fcc-dd2250a20eb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-a2c36d62-0bcd-457b-9fcc-dd2250a20eb2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:43.793421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:43.794454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:43.795331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:43.795331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:43.795331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:43.795331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:43.795331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:43.795331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:43.795331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:43.795331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:43.795331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:43.795331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:43.795331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:38:43.795331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:43.799047 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 541/1062] 10.222.0.46 () {56 vars in 1120 bytes} [Fri Sep 9 12:38:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:43.800885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08895f5-bab1-4bfd-8104-91e07fe4d905 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:43.885820 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:43.905318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08895f5-bab1-4bfd-8104-91e07fe4d905 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:43.916530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d1afc-5e28-463a-9717-bf1453e7056c None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:43.916818 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d1afc-5e28-463a-9717-bf1453e7056c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:43.917098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d1afc-5e28-463a-9717-bf1453e7056c None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:43.918607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 542/1063] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:38:43.923186 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8790f-84c3-4bf3-afae-9525bb357fec None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:43.923470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8790f-84c3-4bf3-afae-9525bb357fec None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:43.923744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8790f-84c3-4bf3-afae-9525bb357fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:43.940706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff508cb8-d3ad-41c1-ac0a-979956d4f825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:43.944883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff508cb8-d3ad-41c1-ac0a-979956d4f825 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:43.944883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff508cb8-d3ad-41c1-ac0a-979956d4f825 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:43.944883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff508cb8-d3ad-41c1-ac0a-979956d4f825 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:43.944883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff508cb8-d3ad-41c1-ac0a-979956d4f825 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:43.945149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d9d8790f-84c3-4bf3-afae-9525bb357fec None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:43.945241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff508cb8-d3ad-41c1-ac0a-979956d4f825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:43.945314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d9d8790f-84c3-4bf3-afae-9525bb357fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:43.945383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff508cb8-d3ad-41c1-ac0a-979956d4f825 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:43.952794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 522/1064] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 12:38:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:43.958176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff508cb8-d3ad-41c1-ac0a-979956d4f825 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:43.965217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:44.015108 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 543/1065] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:44.071563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff508cb8-d3ad-41c1-ac0a-979956d4f825 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:44.109260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0ba62-dc29-4a0d-87d6-3cb3bd68acc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:44.109977 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0ba62-dc29-4a0d-87d6-3cb3bd68acc9 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:44.110258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0ba62-dc29-4a0d-87d6-3cb3bd68acc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:44.110562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0ba62-dc29-4a0d-87d6-3cb3bd68acc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:44.129053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af0ba62-dc29-4a0d-87d6-3cb3bd68acc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:44.130517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af0ba62-dc29-4a0d-87d6-3cb3bd68acc9 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:44.133700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-8af0ba62-dc29-4a0d-87d6-3cb3bd68acc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:44.148198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:44.149163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:44.150104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:44.150104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:44.150104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:44.151062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 523/1066] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:44.155054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8790f-84c3-4bf3-afae-9525bb357fec None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:44.257429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8790f-84c3-4bf3-afae-9525bb357fec None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:44.291706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61414060-6b82-45f5-8a24-289c00c226f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:44.292314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61414060-6b82-45f5-8a24-289c00c226f9 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:44.292588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61414060-6b82-45f5-8a24-289c00c226f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:44.292853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61414060-6b82-45f5-8a24-289c00c226f9 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:44.293587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61414060-6b82-45f5-8a24-289c00c226f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:44.293883 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61414060-6b82-45f5-8a24-289c00c226f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:44.294723 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0ba62-dc29-4a0d-87d6-3cb3bd68acc9 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:44.295243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61414060-6b82-45f5-8a24-289c00c226f9 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:44.309247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 544/1067] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:44.315071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61414060-6b82-45f5-8a24-289c00c226f9 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:44.336149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:44.404402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0ba62-dc29-4a0d-87d6-3cb3bd68acc9 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:44.448775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a77df-f30a-4ffd-a09e-6ac7e681ae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:44.449405 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a77df-f30a-4ffd-a09e-6ac7e681ae44 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:44.449685 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a77df-f30a-4ffd-a09e-6ac7e681ae44 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:44.449952 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a77df-f30a-4ffd-a09e-6ac7e681ae44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:44.455971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61414060-6b82-45f5-8a24-289c00c226f9 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:44.461302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a77df-f30a-4ffd-a09e-6ac7e681ae44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-d94a77df-f30a-4ffd-a09e-6ac7e681ae44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:44.465979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:44.466993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:44.467988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:44.467988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:44.467988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:44.467988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:44.467988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:44.467988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:44.467988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:44.467988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:44.467988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:44.467988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:44.467988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:38:44.467988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:44.467988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 524/1068] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:38:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:44.472961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a77df-f30a-4ffd-a09e-6ac7e681ae44 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:44.496168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fb2c1-f43b-44f7-bd58-e031a13b31a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:44.496807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fb2c1-f43b-44f7-bd58-e031a13b31a9 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:44.497087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fb2c1-f43b-44f7-bd58-e031a13b31a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:44.497358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fb2c1-f43b-44f7-bd58-e031a13b31a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:44.504949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090fb2c1-f43b-44f7-bd58-e031a13b31a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:44.506666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090fb2c1-f43b-44f7-bd58-e031a13b31a9 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-090fb2c1-f43b-44f7-bd58-e031a13b31a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:44.517746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:44.518935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:44.520067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:44.520067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:44.520067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:44.520067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:44.520067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:44.520067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:44.520067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:44.520067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:44.520067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:38:44.520067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:44.520067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 545/1069] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 12:38:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:44.525425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fb2c1-f43b-44f7-bd58-e031a13b31a9 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:44.533517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:44.582069 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a77df-f30a-4ffd-a09e-6ac7e681ae44 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:44.602582 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:44.619589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269c43f-1b4a-4325-b9eb-8281ec466519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:44.620196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269c43f-1b4a-4325-b9eb-8281ec466519 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:44.620606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269c43f-1b4a-4325-b9eb-8281ec466519 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:44.620887 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269c43f-1b4a-4325-b9eb-8281ec466519 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:44.621622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269c43f-1b4a-4325-b9eb-8281ec466519 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:44.621923 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269c43f-1b4a-4325-b9eb-8281ec466519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:44.623319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269c43f-1b4a-4325-b9eb-8281ec466519 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:44.624851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fb2c1-f43b-44f7-bd58-e031a13b31a9 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:44.631887 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 525/1070] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:38:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:44.637662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269c43f-1b4a-4325-b9eb-8281ec466519 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:44.666005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f8d5d-5968-4ce8-871b-38c265a49d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:44.666575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f8d5d-5968-4ce8-871b-38c265a49d13 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:44.666886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f8d5d-5968-4ce8-871b-38c265a49d13 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:44.667160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f8d5d-5968-4ce8-871b-38c265a49d13 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:44.667906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f8d5d-5968-4ce8-871b-38c265a49d13 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:44.668210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f8d5d-5968-4ce8-871b-38c265a49d13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:44.669660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f8d5d-5968-4ce8-871b-38c265a49d13 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:44.689949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 546/1071] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 12:38:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:44.695720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f8d5d-5968-4ce8-871b-38c265a49d13 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:44.781774 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269c43f-1b4a-4325-b9eb-8281ec466519 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:44.797819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:44.810637 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f8d5d-5968-4ce8-871b-38c265a49d13 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:44.816570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd2903-7951-4636-9f58-b2f0f7ab0ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:44.817197 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd2903-7951-4636-9f58-b2f0f7ab0ca5 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:44.817478 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd2903-7951-4636-9f58-b2f0f7ab0ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:44.817789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd2903-7951-4636-9f58-b2f0f7ab0ca5 None admin] PATH_INFO: `/v3/users/6928aac4953443dfa32b3d40cf7e149f` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:44.819542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd2903-7951-4636-9f58-b2f0f7ab0ca5 None admin] RBAC: Authorizing `identity:get_user(user_id=6928aac4953443dfa32b3d40cf7e149f)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:44.821075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd2903-7951-4636-9f58-b2f0f7ab0ca5 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:44.822675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 526/1072] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:38:44 2022] GET /identity/v3/users/6928aac4953443dfa32b3d40cf7e149f => generated 285 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:44.823790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f663fd-da2e-43c6-9910-30f355cf62de None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:44.824074 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f663fd-da2e-43c6-9910-30f355cf62de None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:44.824348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f663fd-da2e-43c6-9910-30f355cf62de None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:44.825684 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 527/1073] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:44.828976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f663fd-da2e-43c6-9910-30f355cf62de None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:44.846964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6a4e3-b888-4e5d-918a-118052a611c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:44.847590 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6a4e3-b888-4e5d-918a-118052a611c9 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:44.847864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6a4e3-b888-4e5d-918a-118052a611c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:44.848130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6a4e3-b888-4e5d-918a-118052a611c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:44.849979 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:44.858346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6a4e3-b888-4e5d-918a-118052a611c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:44.859799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6a4e3-b888-4e5d-918a-118052a611c9 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-69a6a4e3-b888-4e5d-918a-118052a611c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:44.874065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:44.875247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:44.876415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:44.876415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:44.876415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:44.876415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 547/1074] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:44.876415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af43ca-660f-464a-b099-1da5b49f4856 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:44.876775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af43ca-660f-464a-b099-1da5b49f4856 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:44.876775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af43ca-660f-464a-b099-1da5b49f4856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:44.894135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-72af43ca-660f-464a-b099-1da5b49f4856 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:44.895353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-72af43ca-660f-464a-b099-1da5b49f4856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:44.933795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f663fd-da2e-43c6-9910-30f355cf62de None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:44.960790 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 548/1075] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:44.961615 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af43ca-660f-464a-b099-1da5b49f4856 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:44.972266 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a90f17-f3af-4351-bcf5-4ac1f80324ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:44.972897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90f17-f3af-4351-bcf5-4ac1f80324ca None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:44.973181 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90f17-f3af-4351-bcf5-4ac1f80324ca None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:44.973461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90f17-f3af-4351-bcf5-4ac1f80324ca None admin] PATH_INFO: `/v3/projects/8bfb7d578bd241fa9841964b4d3410e4` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:44.974605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a90f17-f3af-4351-bcf5-4ac1f80324ca None admin] RBAC: Authorizing `identity:get_project(project_id=8bfb7d578bd241fa9841964b4d3410e4)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:44.976569 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a90f17-f3af-4351-bcf5-4ac1f80324ca None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:44.978185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 528/1076] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:38:44 2022] GET /identity/v3/projects/8bfb7d578bd241fa9841964b4d3410e4 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:44.984076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a90f17-f3af-4351-bcf5-4ac1f80324ca None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:45.040595 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:45.074528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af43ca-660f-464a-b099-1da5b49f4856 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:45.089512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a90f17-f3af-4351-bcf5-4ac1f80324ca None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:45.092851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:45.115292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14148441-082e-433b-a2f0-1c7e1977d0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:45.115975 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14148441-082e-433b-a2f0-1c7e1977d0b8 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:45.116246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14148441-082e-433b-a2f0-1c7e1977d0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:45.116843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14148441-082e-433b-a2f0-1c7e1977d0b8 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:45.117206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14148441-082e-433b-a2f0-1c7e1977d0b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:45.117514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14148441-082e-433b-a2f0-1c7e1977d0b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:45.121114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14148441-082e-433b-a2f0-1c7e1977d0b8 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:45.137257 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaadf17-7f90-427d-94fd-e21247a65d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:45.137934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaadf17-7f90-427d-94fd-e21247a65d77 None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:45.138219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaadf17-7f90-427d-94fd-e21247a65d77 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:45.140914 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaadf17-7f90-427d-94fd-e21247a65d77 None admin] PATH_INFO: `/v3/projects/8bfb7d578bd241fa9841964b4d3410e4/users/6928aac4953443dfa32b3d40cf7e149f/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:45.147259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaadf17-7f90-427d-94fd-e21247a65d77 None admin] RBAC: Authorizing `identity:create_grant(project_id=8bfb7d578bd241fa9841964b4d3410e4, user_id=6928aac4953443dfa32b3d40cf7e149f, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:45.147259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaadf17-7f90-427d-94fd-e21247a65d77 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:45.151549 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 549/1077] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:45.156754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14148441-082e-433b-a2f0-1c7e1977d0b8 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:45.226811 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 529/1078] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:38:44 2022] PUT /identity/v3/projects/8bfb7d578bd241fa9841964b4d3410e4/users/6928aac4953443dfa32b3d40cf7e149f/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:38:45.232233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaadf17-7f90-427d-94fd-e21247a65d77 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:45.266593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14148441-082e-433b-a2f0-1c7e1977d0b8 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:45.305291 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb43a32-c731-459a-a4d7-ae1b0843a7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:45.306217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb43a32-c731-459a-a4d7-ae1b0843a7f7 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:45.306519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb43a32-c731-459a-a4d7-ae1b0843a7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:45.306816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb43a32-c731-459a-a4d7-ae1b0843a7f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:45.315084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb43a32-c731-459a-a4d7-ae1b0843a7f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:45.316578 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb43a32-c731-459a-a4d7-ae1b0843a7f7 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-ffb43a32-c731-459a-a4d7-ae1b0843a7f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:45.329308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:45.330330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:45.331598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:45.331598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:45.331598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:45.331598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:45.331598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:45.331598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:45.331598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:45.331598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:45.331598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 550/1079] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:45.336766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb43a32-c731-459a-a4d7-ae1b0843a7f7 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:45.350662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaadf17-7f90-427d-94fd-e21247a65d77 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:45.365138 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:45.377795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:45.396853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1a389-1531-45cb-8faf-a7b5545c21e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:45.397477 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1a389-1531-45cb-8faf-a7b5545c21e0 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:45.397758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1a389-1531-45cb-8faf-a7b5545c21e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:45.398029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1a389-1531-45cb-8faf-a7b5545c21e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:45.455007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb43a32-c731-459a-a4d7-ae1b0843a7f7 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:45.473846 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1a389-1531-45cb-8faf-a7b5545c21e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:45.475526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1a389-1531-45cb-8faf-a7b5545c21e0 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:45.489249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9729660-e793-47b7-9dec-7048f034ac07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:45.489784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9729660-e793-47b7-9dec-7048f034ac07 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:45.490063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9729660-e793-47b7-9dec-7048f034ac07 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:45.490336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9729660-e793-47b7-9dec-7048f034ac07 None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:45.491974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9729660-e793-47b7-9dec-7048f034ac07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:45.492288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9729660-e793-47b7-9dec-7048f034ac07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:45.493700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9729660-e793-47b7-9dec-7048f034ac07 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:45.505633 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 551/1080] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:38:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:45.512614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9729660-e793-47b7-9dec-7048f034ac07 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:45.582283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 530/1081] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:38:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 12:38:45.588817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1a389-1531-45cb-8faf-a7b5545c21e0 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:45.603295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:45.612389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:45.618058 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9729660-e793-47b7-9dec-7048f034ac07 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:45.652878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf5724-7166-4ebe-b974-17dbbf7c1798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:45.653535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf5724-7166-4ebe-b974-17dbbf7c1798 None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:45.653810 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf5724-7166-4ebe-b974-17dbbf7c1798 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:45.654078 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf5724-7166-4ebe-b974-17dbbf7c1798 None admin] PATH_INFO: `/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/f29af98c5fba4569b16edf240b13c5c5/roles/0f979abba91c44569a1ceb70bceceacb` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:45.656283 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf5724-7166-4ebe-b974-17dbbf7c1798 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e2dc716277d42ae8316ebb2d36293a8, user_id=f29af98c5fba4569b16edf240b13c5c5, role_id=0f979abba91c44569a1ceb70bceceacb)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:45.657695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf5724-7166-4ebe-b974-17dbbf7c1798 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:45.691898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1a389-1531-45cb-8faf-a7b5545c21e0 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:45.727656 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a16959-fb9d-49ed-96df-16ed2251f734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:45.728457 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a16959-fb9d-49ed-96df-16ed2251f734 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:45.728740 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a16959-fb9d-49ed-96df-16ed2251f734 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:45.729015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a16959-fb9d-49ed-96df-16ed2251f734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:45.731358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 552/1082] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:38:45 2022] PUT /identity/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/f29af98c5fba4569b16edf240b13c5c5/roles/0f979abba91c44569a1ceb70bceceacb => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:38:45.751828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a16959-fb9d-49ed-96df-16ed2251f734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:45.753360 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a16959-fb9d-49ed-96df-16ed2251f734 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-b7a16959-fb9d-49ed-96df-16ed2251f734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:45.771494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:45.773583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:45.774509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:45.774509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:45.774509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:45.778833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 531/1083] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:45.780052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf5724-7166-4ebe-b974-17dbbf7c1798 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:45.889579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf5724-7166-4ebe-b974-17dbbf7c1798 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:45.927315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633f678-f8a6-42d3-af77-55055ecceb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:45.927969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633f678-f8a6-42d3-af77-55055ecceb6a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:45.928251 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633f678-f8a6-42d3-af77-55055ecceb6a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:45.928532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633f678-f8a6-42d3-af77-55055ecceb6a None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:45.929209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633f678-f8a6-42d3-af77-55055ecceb6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:45.929508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633f678-f8a6-42d3-af77-55055ecceb6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:45.930961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633f678-f8a6-42d3-af77-55055ecceb6a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:45.946613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 553/1084] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:47.397776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb666af-e744-4bf8-a50f-a28ed1779382 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:47.398321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb666af-e744-4bf8-a50f-a28ed1779382 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:47.398321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb666af-e744-4bf8-a50f-a28ed1779382 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:47.399611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 532/1085] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:47.405002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f2f00-c0ce-4aa4-9da8-95648bf87f68 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:47.405406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f2f00-c0ce-4aa4-9da8-95648bf87f68 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:47.405552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f2f00-c0ce-4aa4-9da8-95648bf87f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:47.423831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-637f2f00-c0ce-4aa4-9da8-95648bf87f68 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:47.424998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-637f2f00-c0ce-4aa4-9da8-95648bf87f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:47.436301 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:47.491540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 554/1086] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:47.759557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb666af-e744-4bf8-a50f-a28ed1779382 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:47.826354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:47.864276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb666af-e744-4bf8-a50f-a28ed1779382 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:47.895326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ae85e-3367-427f-8bce-fa46719b2207 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:47.895615 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ae85e-3367-427f-8bce-fa46719b2207 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:47.895873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ae85e-3367-427f-8bce-fa46719b2207 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:47.897482 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 555/1087] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:38:47.897898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddda84e-cc9e-40c0-aa93-e4dec10b181b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:47.898554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddda84e-cc9e-40c0-aa93-e4dec10b181b None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:47.898842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddda84e-cc9e-40c0-aa93-e4dec10b181b None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:47.899120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddda84e-cc9e-40c0-aa93-e4dec10b181b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:47.902175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab0f29-4405-43ba-9d82-e6168b123049 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:47.902455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab0f29-4405-43ba-9d82-e6168b123049 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:47.902752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab0f29-4405-43ba-9d82-e6168b123049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:47.909698 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddda84e-cc9e-40c0-aa93-e4dec10b181b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-0ddda84e-cc9e-40c0-aa93-e4dec10b181b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:47.915334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:47.916489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:47.917390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:47.917390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:47.917390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:47.917390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:47.917390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:47.917390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:47.917390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:47.917390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:47.917390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:47.917390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:47.917390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:38:47.917390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:47.917390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 533/1088] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:38:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:47.922175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddda84e-cc9e-40c0-aa93-e4dec10b181b None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:47.923016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-e0ab0f29-4405-43ba-9d82-e6168b123049 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:47.924195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e0ab0f29-4405-43ba-9d82-e6168b123049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:47.991370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 556/1089] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:48.020681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddda84e-cc9e-40c0-aa93-e4dec10b181b None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:48.043437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:48.053655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d0b00-2628-43fe-a3dc-d0cc872dd23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:48.054264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d0b00-2628-43fe-a3dc-d0cc872dd23a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:48.054565 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d0b00-2628-43fe-a3dc-d0cc872dd23a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:48.054841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d0b00-2628-43fe-a3dc-d0cc872dd23a None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:48.055494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d0b00-2628-43fe-a3dc-d0cc872dd23a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:48.055795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d0b00-2628-43fe-a3dc-d0cc872dd23a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:48.057144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d0b00-2628-43fe-a3dc-d0cc872dd23a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:48.065763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 534/1090] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:38:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:48.072129 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab0f29-4405-43ba-9d82-e6168b123049 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:48.149314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:48.171063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab0f29-4405-43ba-9d82-e6168b123049 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:48.206539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a0faf-967f-4ab0-be98-9903634a8c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:48.207154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a0faf-967f-4ab0-be98-9903634a8c64 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:48.207425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a0faf-967f-4ab0-be98-9903634a8c64 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:48.207692 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a0faf-967f-4ab0-be98-9903634a8c64 None admin] PATH_INFO: `/v3/users/6928aac4953443dfa32b3d40cf7e149f` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:48.208793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a0faf-967f-4ab0-be98-9903634a8c64 None admin] RBAC: Authorizing `identity:get_user(user_id=6928aac4953443dfa32b3d40cf7e149f)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:48.210846 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a0faf-967f-4ab0-be98-9903634a8c64 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:48.212448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 557/1091] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:38:48 2022] GET /identity/v3/users/6928aac4953443dfa32b3d40cf7e149f => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:48.218292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d0b00-2628-43fe-a3dc-d0cc872dd23a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:48.270643 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a0faf-967f-4ab0-be98-9903634a8c64 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:48.316260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d0b00-2628-43fe-a3dc-d0cc872dd23a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:48.349051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfdfee-42f5-454f-8cfd-beaf86146023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:48.349675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfdfee-42f5-454f-8cfd-beaf86146023 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:48.349936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfdfee-42f5-454f-8cfd-beaf86146023 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:48.350223 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfdfee-42f5-454f-8cfd-beaf86146023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:48.370643 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a0faf-967f-4ab0-be98-9903634a8c64 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:48.403536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:48.404016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224da6b-2ccc-418b-92a1-a002ae2e18e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:48.404604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224da6b-2ccc-418b-92a1-a002ae2e18e4 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:48.404879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224da6b-2ccc-418b-92a1-a002ae2e18e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:48.405148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224da6b-2ccc-418b-92a1-a002ae2e18e4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:48.415728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224da6b-2ccc-418b-92a1-a002ae2e18e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:38:48.420216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bfdfee-42f5-454f-8cfd-beaf86146023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:48.420216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bfdfee-42f5-454f-8cfd-beaf86146023 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-7224da6b-2ccc-418b-92a1-a002ae2e18e4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:38:48.420490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:48.421381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:48.422244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:48.422244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:48.422244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:48.422244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:48.422244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:38:48.422244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:38:48.422244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:38:48.422244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:38:48.422244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:38:48.422244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:38:48.422244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:38:48.422244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:48.426672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 558/1092] 10.222.0.46 () {56 vars in 1120 bytes} [Fri Sep 9 12:38:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:48.427488 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224da6b-2ccc-418b-92a1-a002ae2e18e4 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:48.432542 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:48.519629 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 535/1093] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:38:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:38:48.526255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfdfee-42f5-454f-8cfd-beaf86146023 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:48.529004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224da6b-2ccc-418b-92a1-a002ae2e18e4 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:48.563094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9040e2a-e1a5-4cd0-9363-24dcd3a398b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:48.563692 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9040e2a-e1a5-4cd0-9363-24dcd3a398b7 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:48.563967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9040e2a-e1a5-4cd0-9363-24dcd3a398b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:48.564232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9040e2a-e1a5-4cd0-9363-24dcd3a398b7 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:48.564881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9040e2a-e1a5-4cd0-9363-24dcd3a398b7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:48.565517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9040e2a-e1a5-4cd0-9363-24dcd3a398b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:48.566588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9040e2a-e1a5-4cd0-9363-24dcd3a398b7 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:48.574661 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 559/1094] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 12:38:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:48.580396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9040e2a-e1a5-4cd0-9363-24dcd3a398b7 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:48.613744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:48.627808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfdfee-42f5-454f-8cfd-beaf86146023 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:48.644353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:48.662404 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71f7f1-ee61-4855-9152-234da4118642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:48.663052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71f7f1-ee61-4855-9152-234da4118642 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:48.663328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71f7f1-ee61-4855-9152-234da4118642 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:48.663600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71f7f1-ee61-4855-9152-234da4118642 None admin] PATH_INFO: `/v3/projects/8bfb7d578bd241fa9841964b4d3410e4` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:48.664689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71f7f1-ee61-4855-9152-234da4118642 None admin] RBAC: Authorizing `identity:get_project(project_id=8bfb7d578bd241fa9841964b4d3410e4)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:48.666104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71f7f1-ee61-4855-9152-234da4118642 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:48.667768 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 536/1095] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:38:48 2022] GET /identity/v3/projects/8bfb7d578bd241fa9841964b4d3410e4 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:48.674039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71f7f1-ee61-4855-9152-234da4118642 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:48.678107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9040e2a-e1a5-4cd0-9363-24dcd3a398b7 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:48.712931 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee50b2f-4fc8-4642-8ee3-94e0fae0cefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:48.713548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee50b2f-4fc8-4642-8ee3-94e0fae0cefc None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:48.713828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee50b2f-4fc8-4642-8ee3-94e0fae0cefc None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:48.714091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee50b2f-4fc8-4642-8ee3-94e0fae0cefc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:48.724547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee50b2f-4fc8-4642-8ee3-94e0fae0cefc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:48.725971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee50b2f-4fc8-4642-8ee3-94e0fae0cefc None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-dee50b2f-4fc8-4642-8ee3-94e0fae0cefc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:48.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:48.740870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:48.742403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:48.742403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:48.742403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:48.742403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 560/1096] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:48.747342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee50b2f-4fc8-4642-8ee3-94e0fae0cefc None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:48.776055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71f7f1-ee61-4855-9152-234da4118642 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:48.808408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac271c-a515-4772-a676-e26a9a7e8daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:48.809001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac271c-a515-4772-a676-e26a9a7e8daa None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:48.809280 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac271c-a515-4772-a676-e26a9a7e8daa None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:48.809543 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac271c-a515-4772-a676-e26a9a7e8daa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:48.810248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac271c-a515-4772-a676-e26a9a7e8daa None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=8bfb7d578bd241fa9841964b4d3410e4, user.id=6928aac4953443dfa32b3d40cf7e149f) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:48.810586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac271c-a515-4772-a676-e26a9a7e8daa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:48.812266 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac271c-a515-4772-a676-e26a9a7e8daa None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:48.830658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 537/1097] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:38:48 2022] GET /identity/v3/role_assignments?user.id=6928aac4953443dfa32b3d40cf7e149f&scope.project.id=8bfb7d578bd241fa9841964b4d3410e4&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:48.847254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee50b2f-4fc8-4642-8ee3-94e0fae0cefc None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:48.865242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:48.884570 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec96c6-eac3-4a5f-85a1-b256d85728ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:48.885280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec96c6-eac3-4a5f-85a1-b256d85728ba None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:48.885562 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec96c6-eac3-4a5f-85a1-b256d85728ba None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:48.885835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec96c6-eac3-4a5f-85a1-b256d85728ba None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:48.886506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec96c6-eac3-4a5f-85a1-b256d85728ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:48.886821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec96c6-eac3-4a5f-85a1-b256d85728ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:48.888187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec96c6-eac3-4a5f-85a1-b256d85728ba None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:48.902433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 561/1098] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:48.907913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac271c-a515-4772-a676-e26a9a7e8daa None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:48.928751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:49.014144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac271c-a515-4772-a676-e26a9a7e8daa None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:49.047158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65c379-6e39-4b82-a5b7-a661f0224ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:49.047775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65c379-6e39-4b82-a5b7-a661f0224ca1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:49.048053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65c379-6e39-4b82-a5b7-a661f0224ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:49.048322 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65c379-6e39-4b82-a5b7-a661f0224ca1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:49.056497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65c379-6e39-4b82-a5b7-a661f0224ca1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:49.057917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65c379-6e39-4b82-a5b7-a661f0224ca1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-0b65c379-6e39-4b82-a5b7-a661f0224ca1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:38:49.069012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:49.070124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:49.071304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:49.071304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:49.071304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:38:49.071304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:38:49.071304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:38:49.071304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:38:49.071304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:38:49.071304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:38:49.071304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:38:49.071304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:38:49.071304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 538/1099] 10.222.0.46 () {56 vars in 1093 bytes} [Fri Sep 9 12:38:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:49.075997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec96c6-eac3-4a5f-85a1-b256d85728ba None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:49.180126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec96c6-eac3-4a5f-85a1-b256d85728ba None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:49.214907 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d29c5-8c7e-4309-b511-a86aa914a7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:49.215516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d29c5-8c7e-4309-b511-a86aa914a7eb None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:49.215792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d29c5-8c7e-4309-b511-a86aa914a7eb None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:49.216063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d29c5-8c7e-4309-b511-a86aa914a7eb None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:49.216785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d29c5-8c7e-4309-b511-a86aa914a7eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:49.217085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d29c5-8c7e-4309-b511-a86aa914a7eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:49.218509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d29c5-8c7e-4309-b511-a86aa914a7eb None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:49.235794 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 562/1100] 10.222.0.46 () {56 vars in 1133 bytes} [Fri Sep 9 12:38:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:49.241853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65c379-6e39-4b82-a5b7-a661f0224ca1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:49.320811 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:49.340991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65c379-6e39-4b82-a5b7-a661f0224ca1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:49.373660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f68986-189c-4154-924c-afedec4ac54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:49.374186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f68986-189c-4154-924c-afedec4ac54d None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:49.374502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f68986-189c-4154-924c-afedec4ac54d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:49.374784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f68986-189c-4154-924c-afedec4ac54d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:49.439543 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f68986-189c-4154-924c-afedec4ac54d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:49.440968 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f68986-189c-4154-924c-afedec4ac54d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:49.513563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:49.539741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 539/1101] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:38:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:38:49.541815 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d29c5-8c7e-4309-b511-a86aa914a7eb None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:49.576265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:49.639357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d29c5-8c7e-4309-b511-a86aa914a7eb None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:49.671616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffe354-1870-4735-9975-f179a3ca6c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:49.672234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffe354-1870-4735-9975-f179a3ca6c6b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:49.672506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffe354-1870-4735-9975-f179a3ca6c6b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:49.672771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffe354-1870-4735-9975-f179a3ca6c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:49.682957 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffe354-1870-4735-9975-f179a3ca6c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:49.684368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffe354-1870-4735-9975-f179a3ca6c6b None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-16ffe354-1870-4735-9975-f179a3ca6c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:38:49.698557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:49.704552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:38:49.706005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:38:49.706005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:38:49.706005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:49.706005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 563/1102] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:38:49.706289 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f68986-189c-4154-924c-afedec4ac54d None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:49.802506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f68986-189c-4154-924c-afedec4ac54d None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:49.835009 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee94c8-1564-408a-b947-16fd44fb16f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:49.835693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee94c8-1564-408a-b947-16fd44fb16f6 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:49.835971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee94c8-1564-408a-b947-16fd44fb16f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:49.836240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee94c8-1564-408a-b947-16fd44fb16f6 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:49.836865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee94c8-1564-408a-b947-16fd44fb16f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:49.837164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee94c8-1564-408a-b947-16fd44fb16f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:49.838533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee94c8-1564-408a-b947-16fd44fb16f6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:49.849067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 540/1103] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:49.855428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffe354-1870-4735-9975-f179a3ca6c6b None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:49.941326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:49.952820 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffe354-1870-4735-9975-f179a3ca6c6b None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:49.986071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e766c36-89d3-40fc-8b4a-252d775bfe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:49.986726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e766c36-89d3-40fc-8b4a-252d775bfe78 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:49.987008 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e766c36-89d3-40fc-8b4a-252d775bfe78 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:49.987280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e766c36-89d3-40fc-8b4a-252d775bfe78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:49.993713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e766c36-89d3-40fc-8b4a-252d775bfe78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:49.995173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e766c36-89d3-40fc-8b4a-252d775bfe78 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-6e766c36-89d3-40fc-8b4a-252d775bfe78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:38:50.006091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:38:50.007024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:38:50.007945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:38:50.007945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:38:50.007945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:38:50.007945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:38:50.007945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:38:50.007945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:38:50.007945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:38:50.007945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:38:50.007945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 564/1104] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:38:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:38:50.013150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee94c8-1564-408a-b947-16fd44fb16f6 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:50.053438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:50.114631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee94c8-1564-408a-b947-16fd44fb16f6 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:50.147885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5a605-5669-46ed-8508-25785fa76e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:50.148484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5a605-5669-46ed-8508-25785fa76e19 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:50.148761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5a605-5669-46ed-8508-25785fa76e19 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:50.149029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5a605-5669-46ed-8508-25785fa76e19 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:50.149699 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5a605-5669-46ed-8508-25785fa76e19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:50.149996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5a605-5669-46ed-8508-25785fa76e19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:50.151434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5a605-5669-46ed-8508-25785fa76e19 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:50.163446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 541/1105] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:38:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:50.169556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e766c36-89d3-40fc-8b4a-252d775bfe78 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:50.266334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e766c36-89d3-40fc-8b4a-252d775bfe78 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:50.298537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de61ea8-fcd0-4dc7-bffc-212f8ccd30ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:50.299125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de61ea8-fcd0-4dc7-bffc-212f8ccd30ce None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:50.299398 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de61ea8-fcd0-4dc7-bffc-212f8ccd30ce None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:50.299666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de61ea8-fcd0-4dc7-bffc-212f8ccd30ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:50.300375 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de61ea8-fcd0-4dc7-bffc-212f8ccd30ce None admin] RBAC: Adding query filter params (role.id=0f979abba91c44569a1ceb70bceceacb, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=f29af98c5fba4569b16edf240b13c5c5) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:50.300674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de61ea8-fcd0-4dc7-bffc-212f8ccd30ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:50.302064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de61ea8-fcd0-4dc7-bffc-212f8ccd30ce None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:50.307705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 565/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:38:50 2022] GET /identity/v3/role_assignments?user.id=f29af98c5fba4569b16edf240b13c5c5&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=0f979abba91c44569a1ceb70bceceacb => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:51.003704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeac443-e295-4ee0-bddc-de01c1639cea None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:51.004244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeac443-e295-4ee0-bddc-de01c1639cea None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:51.004244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeac443-e295-4ee0-bddc-de01c1639cea None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:51.005559 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 542/1107] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:51.011670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39dde4-8725-433c-bf80-1eebb937f0b4 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:51.012092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39dde4-8725-433c-bf80-1eebb937f0b4 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:51.012207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39dde4-8725-433c-bf80-1eebb937f0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:51.030598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-7f39dde4-8725-433c-bf80-1eebb937f0b4 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:51.031779 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7f39dde4-8725-433c-bf80-1eebb937f0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:51.040079 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:51.104166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 566/1108] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:51.377467 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeac443-e295-4ee0-bddc-de01c1639cea None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:51.471267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:51.476428 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeac443-e295-4ee0-bddc-de01c1639cea None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:51.509352 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d210594-ad96-4a83-8fab-930da436d275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:51.509967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d210594-ad96-4a83-8fab-930da436d275 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:51.510247 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d210594-ad96-4a83-8fab-930da436d275 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:51.510543 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d210594-ad96-4a83-8fab-930da436d275 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:51.511639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d210594-ad96-4a83-8fab-930da436d275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:51.513056 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d210594-ad96-4a83-8fab-930da436d275 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:51.514613 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 543/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:38:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:51.520365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39dde4-8725-433c-bf80-1eebb937f0b4 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:51.614756 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39dde4-8725-433c-bf80-1eebb937f0b4 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:51.650192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16556b95-1c86-4f98-9465-440b98b941f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:51.650755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16556b95-1c86-4f98-9465-440b98b941f1 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:51.651027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16556b95-1c86-4f98-9465-440b98b941f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:51.651291 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16556b95-1c86-4f98-9465-440b98b941f1 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:51.651936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16556b95-1c86-4f98-9465-440b98b941f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:51.653319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16556b95-1c86-4f98-9465-440b98b941f1 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:51.753548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:51.759334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 567/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:38:51 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:38:53.954465 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ca09c-efaf-4816-a516-df69d40a894f None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:53.955031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ca09c-efaf-4816-a516-df69d40a894f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:53.955031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ca09c-efaf-4816-a516-df69d40a894f None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:53.956370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 544/1111] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:53.962161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f1c94-35b5-48fb-a0d9-94df45d9fb53 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:53.962614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f1c94-35b5-48fb-a0d9-94df45d9fb53 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:53.962875 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f1c94-35b5-48fb-a0d9-94df45d9fb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:53.980516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-a62f1c94-35b5-48fb-a0d9-94df45d9fb53 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:53.981722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a62f1c94-35b5-48fb-a0d9-94df45d9fb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:54.044854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 568/1112] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:38:54.333009 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ca09c-efaf-4816-a516-df69d40a894f None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:54.431616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ca09c-efaf-4816-a516-df69d40a894f None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:54.477697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239a85e-1d4c-4ac3-824d-6f7c768689dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:54.478341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239a85e-1d4c-4ac3-824d-6f7c768689dd None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:54.478655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239a85e-1d4c-4ac3-824d-6f7c768689dd None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:54.478930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239a85e-1d4c-4ac3-824d-6f7c768689dd None admin] PATH_INFO: `/v3/roles/96271a2245c144d4b43dea66f18074e8` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:54.480388 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e239a85e-1d4c-4ac3-824d-6f7c768689dd None admin] RBAC: Authorizing `identity:get_role(role_id=96271a2245c144d4b43dea66f18074e8)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:54.481887 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e239a85e-1d4c-4ac3-824d-6f7c768689dd None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:54.483293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 545/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:38:54 2022] GET /identity/v3/roles/96271a2245c144d4b43dea66f18074e8 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:54.489102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f1c94-35b5-48fb-a0d9-94df45d9fb53 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:54.585193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f1c94-35b5-48fb-a0d9-94df45d9fb53 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:54.610131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:54.617740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b2023-d5a7-4566-bb9e-82149138c34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:54.618356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b2023-d5a7-4566-bb9e-82149138c34a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:54.618670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b2023-d5a7-4566-bb9e-82149138c34a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:54.618939 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b2023-d5a7-4566-bb9e-82149138c34a None admin] PATH_INFO: `/v3/users/ed6fdae962904662944d46629208daf8` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:54.631969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b2023-d5a7-4566-bb9e-82149138c34a None admin] RBAC: Authorizing `identity:get_user(user_id=ed6fdae962904662944d46629208daf8)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:54.633558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b2023-d5a7-4566-bb9e-82149138c34a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:54.635155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 569/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:38:54 2022] GET /identity/v3/users/ed6fdae962904662944d46629208daf8 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:54.640880 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239a85e-1d4c-4ac3-824d-6f7c768689dd None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:54.674160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:54.737471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239a85e-1d4c-4ac3-824d-6f7c768689dd None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:54.770981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268e58-f1a5-4cf7-b1db-754ee45bf0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:54.772214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268e58-f1a5-4cf7-b1db-754ee45bf0f5 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:54.772492 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268e58-f1a5-4cf7-b1db-754ee45bf0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:54.772762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268e58-f1a5-4cf7-b1db-754ee45bf0f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:54.845855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2268e58-f1a5-4cf7-b1db-754ee45bf0f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:54.847255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2268e58-f1a5-4cf7-b1db-754ee45bf0f5 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:54.879634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:54.943001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 546/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:38:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:38:54.949437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b2023-d5a7-4566-bb9e-82149138c34a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:55.047242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b2023-d5a7-4566-bb9e-82149138c34a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:55.080814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964af50f-dbe4-4f55-9b87-ce57c37af492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:55.081423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964af50f-dbe4-4f55-9b87-ce57c37af492 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:55.081698 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964af50f-dbe4-4f55-9b87-ce57c37af492 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:55.081967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964af50f-dbe4-4f55-9b87-ce57c37af492 None admin] PATH_INFO: `/v3/projects/8bfb7d578bd241fa9841964b4d3410e4` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:55.083069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964af50f-dbe4-4f55-9b87-ce57c37af492 None admin] RBAC: Authorizing `identity:get_project(project_id=8bfb7d578bd241fa9841964b4d3410e4)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:55.084551 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964af50f-dbe4-4f55-9b87-ce57c37af492 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:55.086137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 570/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:38:54 2022] GET /identity/v3/projects/8bfb7d578bd241fa9841964b4d3410e4 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:55.092490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268e58-f1a5-4cf7-b1db-754ee45bf0f5 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:55.200240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268e58-f1a5-4cf7-b1db-754ee45bf0f5 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:55.234841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef85cb-2ced-4b3a-9941-b39364d3d29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:55.235437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef85cb-2ced-4b3a-9941-b39364d3d29d None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:55.235717 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef85cb-2ced-4b3a-9941-b39364d3d29d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:55.235986 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef85cb-2ced-4b3a-9941-b39364d3d29d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:55.236693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef85cb-2ced-4b3a-9941-b39364d3d29d None admin] RBAC: Adding query filter params (role.id=96271a2245c144d4b43dea66f18074e8, scope.project.id=8bfb7d578bd241fa9841964b4d3410e4, user.id=ed6fdae962904662944d46629208daf8) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:38:55.236995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef85cb-2ced-4b3a-9941-b39364d3d29d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:55.238408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef85cb-2ced-4b3a-9941-b39364d3d29d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:55.240126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:55.244102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 547/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:38:55 2022] GET /identity/v3/role_assignments?user.id=ed6fdae962904662944d46629208daf8&scope.project.id=8bfb7d578bd241fa9841964b4d3410e4&role.id=96271a2245c144d4b43dea66f18074e8 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:57.334833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c80cfd-1841-4520-afa5-a0b8ad09111e None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:57.335357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c80cfd-1841-4520-afa5-a0b8ad09111e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:57.335532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c80cfd-1841-4520-afa5-a0b8ad09111e None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:57.336869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 571/1118] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:38:57.341935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f00b8-0ff2-4505-a8d2-252054240f94 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:57.342332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f00b8-0ff2-4505-a8d2-252054240f94 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:57.342572 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f00b8-0ff2-4505-a8d2-252054240f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:57.360812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-cb1f00b8-0ff2-4505-a8d2-252054240f94 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:38:57.361991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-cb1f00b8-0ff2-4505-a8d2-252054240f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:38:57.426085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 548/1119] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:38:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:38:57.700334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c80cfd-1841-4520-afa5-a0b8ad09111e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:57.755307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:57.799521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c80cfd-1841-4520-afa5-a0b8ad09111e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:57.832603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19851082-79e0-4d7c-8f67-a38967746a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:57.833212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19851082-79e0-4d7c-8f67-a38967746a49 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:57.833493 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19851082-79e0-4d7c-8f67-a38967746a49 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:57.833767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19851082-79e0-4d7c-8f67-a38967746a49 None admin] PATH_INFO: `/v3/roles/96271a2245c144d4b43dea66f18074e8` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:57.835751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19851082-79e0-4d7c-8f67-a38967746a49 None admin] RBAC: Authorizing `identity:get_role(role_id=96271a2245c144d4b43dea66f18074e8)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:57.837087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19851082-79e0-4d7c-8f67-a38967746a49 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:57.838389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 572/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:38:57 2022] GET /identity/v3/roles/96271a2245c144d4b43dea66f18074e8 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:57.844063 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f00b8-0ff2-4505-a8d2-252054240f94 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:57.939816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f00b8-0ff2-4505-a8d2-252054240f94 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:57.965892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:57.973391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d427fa8-780b-4bf5-a50e-34a8c8a289a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:57.977296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d427fa8-780b-4bf5-a50e-34a8c8a289a9 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:57.977296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d427fa8-780b-4bf5-a50e-34a8c8a289a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:57.977296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d427fa8-780b-4bf5-a50e-34a8c8a289a9 None admin] PATH_INFO: `/v3/users/ed6fdae962904662944d46629208daf8` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:57.977296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d427fa8-780b-4bf5-a50e-34a8c8a289a9 None admin] RBAC: Authorizing `identity:get_user(user_id=ed6fdae962904662944d46629208daf8)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:57.977296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d427fa8-780b-4bf5-a50e-34a8c8a289a9 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:57.978763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 549/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:38:57 2022] GET /identity/v3/users/ed6fdae962904662944d46629208daf8 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:38:57.984779 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19851082-79e0-4d7c-8f67-a38967746a49 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:58.081020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19851082-79e0-4d7c-8f67-a38967746a49 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:58.102938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:38:58.113535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a585cb-13be-4b20-862c-49eb82fca2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:58.114597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a585cb-13be-4b20-862c-49eb82fca2eb None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:58.114597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a585cb-13be-4b20-862c-49eb82fca2eb None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:58.114778 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a585cb-13be-4b20-862c-49eb82fca2eb None admin] PATH_INFO: `/v3/projects/8bfb7d578bd241fa9841964b4d3410e4` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:58.116363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a585cb-13be-4b20-862c-49eb82fca2eb None admin] RBAC: Authorizing `identity:get_project(project_id=8bfb7d578bd241fa9841964b4d3410e4)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:58.117755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a585cb-13be-4b20-862c-49eb82fca2eb None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:58.119423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 573/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:38:57 2022] GET /identity/v3/projects/8bfb7d578bd241fa9841964b4d3410e4 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:38:58.125221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d427fa8-780b-4bf5-a50e-34a8c8a289a9 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:38:58.222266 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d427fa8-780b-4bf5-a50e-34a8c8a289a9 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:38:58.254759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590763fb-580d-4bb7-8f23-128b89b9a48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:38:58.255397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590763fb-580d-4bb7-8f23-128b89b9a48b None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:38:58.255677 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590763fb-580d-4bb7-8f23-128b89b9a48b None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:38:58.256160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590763fb-580d-4bb7-8f23-128b89b9a48b None admin] PATH_INFO: `/v3/projects/8bfb7d578bd241fa9841964b4d3410e4/users/ed6fdae962904662944d46629208daf8/roles/96271a2245c144d4b43dea66f18074e8` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:38:58.258431 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590763fb-580d-4bb7-8f23-128b89b9a48b None admin] RBAC: Authorizing `identity:create_grant(project_id=8bfb7d578bd241fa9841964b4d3410e4, user_id=ed6fdae962904662944d46629208daf8, role_id=96271a2245c144d4b43dea66f18074e8)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:38:58.259872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590763fb-580d-4bb7-8f23-128b89b9a48b None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:38:58.358957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 550/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:38:58 2022] PUT /identity/v3/projects/8bfb7d578bd241fa9841964b4d3410e4/users/ed6fdae962904662944d46629208daf8/roles/96271a2245c144d4b43dea66f18074e8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:39:00.501928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266443a-8c46-44ce-a69d-349f7be6c187 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:00.502469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266443a-8c46-44ce-a69d-349f7be6c187 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:00.502469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266443a-8c46-44ce-a69d-349f7be6c187 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:00.503825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 574/1124] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:39:00.509497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656af003-88a9-4525-86a9-779216b76862 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:00.509925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656af003-88a9-4525-86a9-779216b76862 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:00.510043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656af003-88a9-4525-86a9-779216b76862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:00.528375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-656af003-88a9-4525-86a9-779216b76862 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:39:00.529530 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-656af003-88a9-4525-86a9-779216b76862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:39:00.577022 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:00.592892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 551/1125] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:39:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:39:00.858684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266443a-8c46-44ce-a69d-349f7be6c187 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:00.954940 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266443a-8c46-44ce-a69d-349f7be6c187 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:00.986019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db8048-f172-4eea-9d20-9c62a8d4b30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:00.986648 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db8048-f172-4eea-9d20-9c62a8d4b30f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:00.986925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db8048-f172-4eea-9d20-9c62a8d4b30f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:00.987195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db8048-f172-4eea-9d20-9c62a8d4b30f None admin] PATH_INFO: `/v3/roles/96271a2245c144d4b43dea66f18074e8` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:00.988540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db8048-f172-4eea-9d20-9c62a8d4b30f None admin] RBAC: Authorizing `identity:get_role(role_id=96271a2245c144d4b43dea66f18074e8)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:00.989872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db8048-f172-4eea-9d20-9c62a8d4b30f None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:00.991719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 575/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:39:00 2022] GET /identity/v3/roles/96271a2245c144d4b43dea66f18074e8 => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:00.997449 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656af003-88a9-4525-86a9-779216b76862 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:01.093070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656af003-88a9-4525-86a9-779216b76862 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:01.126577 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302a0ff-7b93-4aa3-976b-f0397f417b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:01.127211 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302a0ff-7b93-4aa3-976b-f0397f417b4b None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:01.127494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302a0ff-7b93-4aa3-976b-f0397f417b4b None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:01.127767 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302a0ff-7b93-4aa3-976b-f0397f417b4b None admin] PATH_INFO: `/v3/users/ed6fdae962904662944d46629208daf8` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:01.128925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302a0ff-7b93-4aa3-976b-f0397f417b4b None admin] RBAC: Authorizing `identity:get_user(user_id=ed6fdae962904662944d46629208daf8)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:01.130349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302a0ff-7b93-4aa3-976b-f0397f417b4b None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:01.132083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 552/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:39:00 2022] GET /identity/v3/users/ed6fdae962904662944d46629208daf8 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:01.137792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db8048-f172-4eea-9d20-9c62a8d4b30f None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:01.195140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:01.234103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db8048-f172-4eea-9d20-9c62a8d4b30f None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:01.267363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfc179-01a7-4c6f-abaa-ebcb9404ccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:01.267954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfc179-01a7-4c6f-abaa-ebcb9404ccd8 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:01.268229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfc179-01a7-4c6f-abaa-ebcb9404ccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:01.268495 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfc179-01a7-4c6f-abaa-ebcb9404ccd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:01.332988 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfc179-01a7-4c6f-abaa-ebcb9404ccd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:01.334432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfc179-01a7-4c6f-abaa-ebcb9404ccd8 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:01.389086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:01.450165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 576/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:39:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:39:01.459282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302a0ff-7b93-4aa3-976b-f0397f417b4b None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:01.536087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:01.579314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302a0ff-7b93-4aa3-976b-f0397f417b4b None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:01.612582 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6178bbc-9973-4469-9dab-08034429aa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:01.613203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6178bbc-9973-4469-9dab-08034429aa00 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:01.613486 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6178bbc-9973-4469-9dab-08034429aa00 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:01.613762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6178bbc-9973-4469-9dab-08034429aa00 None admin] PATH_INFO: `/v3/projects/8bfb7d578bd241fa9841964b4d3410e4` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:01.614890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6178bbc-9973-4469-9dab-08034429aa00 None admin] RBAC: Authorizing `identity:get_project(project_id=8bfb7d578bd241fa9841964b4d3410e4)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:01.616368 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6178bbc-9973-4469-9dab-08034429aa00 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:01.617979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 553/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:39:01 2022] GET /identity/v3/projects/8bfb7d578bd241fa9841964b4d3410e4 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:01.624272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfc179-01a7-4c6f-abaa-ebcb9404ccd8 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:01.719695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfc179-01a7-4c6f-abaa-ebcb9404ccd8 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:01.752305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35265238-38f2-40e3-9f76-1cbd07e04e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:01.752888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35265238-38f2-40e3-9f76-1cbd07e04e25 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:01.753167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35265238-38f2-40e3-9f76-1cbd07e04e25 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:01.753438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35265238-38f2-40e3-9f76-1cbd07e04e25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:01.754154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35265238-38f2-40e3-9f76-1cbd07e04e25 None admin] RBAC: Adding query filter params (role.id=96271a2245c144d4b43dea66f18074e8, scope.project.id=8bfb7d578bd241fa9841964b4d3410e4, user.id=ed6fdae962904662944d46629208daf8) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:39:01.754460 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35265238-38f2-40e3-9f76-1cbd07e04e25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:01.756439 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35265238-38f2-40e3-9f76-1cbd07e04e25 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:01.762052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 577/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:39:01 2022] GET /identity/v3/role_assignments?user.id=ed6fdae962904662944d46629208daf8&scope.project.id=8bfb7d578bd241fa9841964b4d3410e4&role.id=96271a2245c144d4b43dea66f18074e8 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:03.938569 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03dc57-38cf-44c3-bff2-5cbf14f17d6b None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:03.939133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03dc57-38cf-44c3-bff2-5cbf14f17d6b None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:03.939133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03dc57-38cf-44c3-bff2-5cbf14f17d6b None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:03.940448 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 554/1131] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:39:03.946647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d027ac-2cee-479d-bbf4-6671afb41408 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:03.947066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d027ac-2cee-479d-bbf4-6671afb41408 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:03.947193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d027ac-2cee-479d-bbf4-6671afb41408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:03.965237 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-67d027ac-2cee-479d-bbf4-6671afb41408 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:39:03.966406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-67d027ac-2cee-479d-bbf4-6671afb41408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:39:03.982336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:04.031309 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 578/1132] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:39:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:39:04.307374 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03dc57-38cf-44c3-bff2-5cbf14f17d6b None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:04.408051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03dc57-38cf-44c3-bff2-5cbf14f17d6b None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:04.430866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:04.441873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b92430-a655-4525-8baf-c000e0988765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:04.445169 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b92430-a655-4525-8baf-c000e0988765 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:04.445169 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b92430-a655-4525-8baf-c000e0988765 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:04.445169 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b92430-a655-4525-8baf-c000e0988765 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:04.445169 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b92430-a655-4525-8baf-c000e0988765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:04.445823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b92430-a655-4525-8baf-c000e0988765 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:04.447393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 555/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:39:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:04.453200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d027ac-2cee-479d-bbf4-6671afb41408 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:04.549499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d027ac-2cee-479d-bbf4-6671afb41408 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:04.581718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54430bc-95f0-4970-ad55-c3b7e470afd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:04.582247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54430bc-95f0-4970-ad55-c3b7e470afd0 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:04.582551 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54430bc-95f0-4970-ad55-c3b7e470afd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:04.582824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54430bc-95f0-4970-ad55-c3b7e470afd0 None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:04.583470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54430bc-95f0-4970-ad55-c3b7e470afd0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:04.585373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54430bc-95f0-4970-ad55-c3b7e470afd0 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:04.679855 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 579/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:39:04 2022] POST /identity/v3/projects => generated 311 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:39:06.888496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f932b-1935-4212-a30c-3c76b806f9be None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:06.889048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f932b-1935-4212-a30c-3c76b806f9be None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:06.889048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f932b-1935-4212-a30c-3c76b806f9be None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:06.890362 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 556/1135] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:39:06.895476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec67e3-4576-4120-af78-e76c71e03f12 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:06.895911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec67e3-4576-4120-af78-e76c71e03f12 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:06.896013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec67e3-4576-4120-af78-e76c71e03f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:06.903275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:06.913984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-78ec67e3-4576-4120-af78-e76c71e03f12 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:39:06.915250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-78ec67e3-4576-4120-af78-e76c71e03f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:39:06.980775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 580/1136] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:39:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:39:07.249366 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f932b-1935-4212-a30c-3c76b806f9be None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:07.353410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f932b-1935-4212-a30c-3c76b806f9be None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:07.385790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfdb60-ccf2-419b-831a-e4a3937ecba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:07.386396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfdb60-ccf2-419b-831a-e4a3937ecba8 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:07.386701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfdb60-ccf2-419b-831a-e4a3937ecba8 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:07.386973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfdb60-ccf2-419b-831a-e4a3937ecba8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:07.388068 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfdb60-ccf2-419b-831a-e4a3937ecba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:07.389479 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfdb60-ccf2-419b-831a-e4a3937ecba8 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:07.391050 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 557/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:39:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:07.397460 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec67e3-4576-4120-af78-e76c71e03f12 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:07.492030 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec67e3-4576-4120-af78-e76c71e03f12 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:07.524130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c647c76-7f4e-4332-8916-b61824532c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:07.524670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c647c76-7f4e-4332-8916-b61824532c5f None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:07.524947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c647c76-7f4e-4332-8916-b61824532c5f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:07.525218 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c647c76-7f4e-4332-8916-b61824532c5f None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:07.525871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c647c76-7f4e-4332-8916-b61824532c5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:07.527310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c647c76-7f4e-4332-8916-b61824532c5f None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:07.535123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:07.621370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 581/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:39:07 2022] POST /identity/v3/users => generated 286 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:39:09.847141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee4f20-6748-4d5f-b33a-2fc87acc9ef5 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:09.847675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee4f20-6748-4d5f-b33a-2fc87acc9ef5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:09.847675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee4f20-6748-4d5f-b33a-2fc87acc9ef5 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:09.848997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 558/1139] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:39:09.855216 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7852d-2b84-4bbd-896d-468639726d9d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:09.855605 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7852d-2b84-4bbd-896d-468639726d9d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:09.855758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7852d-2b84-4bbd-896d-468639726d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:09.873883 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-bcf7852d-2b84-4bbd-896d-468639726d9d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:39:09.875082 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-bcf7852d-2b84-4bbd-896d-468639726d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:39:09.939583 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 582/1140] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:39:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:39:10.220386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee4f20-6748-4d5f-b33a-2fc87acc9ef5 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:10.277877 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:10.319771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee4f20-6748-4d5f-b33a-2fc87acc9ef5 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:10.352294 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94708c56-0154-4ff4-a372-d7ff12132fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:10.353141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94708c56-0154-4ff4-a372-d7ff12132fd9 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:10.353599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94708c56-0154-4ff4-a372-d7ff12132fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:10.354342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94708c56-0154-4ff4-a372-d7ff12132fd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:10.364032 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94708c56-0154-4ff4-a372-d7ff12132fd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-94708c56-0154-4ff4-a372-d7ff12132fd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:39:10.370614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:39:10.371576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:39:10.372564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:39:10.372564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:39:10.372564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:39:10.372564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:39:10.372564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:39:10.372564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:39:10.372564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:39:10.372564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:39:10.372564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:39:10.372564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:39:10.372564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:39:10.372564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:39:10.372564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 559/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:39:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:39:10.377736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7852d-2b84-4bbd-896d-468639726d9d None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:10.474251 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7852d-2b84-4bbd-896d-468639726d9d None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:10.494206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:10.507950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e9995-e4a5-4680-aa89-233bb3ee232b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:10.508544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e9995-e4a5-4680-aa89-233bb3ee232b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:10.508816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e9995-e4a5-4680-aa89-233bb3ee232b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:10.509084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e9995-e4a5-4680-aa89-233bb3ee232b None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:10.509736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e9995-e4a5-4680-aa89-233bb3ee232b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:39:10.510029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e9995-e4a5-4680-aa89-233bb3ee232b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:10.511397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e9995-e4a5-4680-aa89-233bb3ee232b None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:10.519062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 583/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:39:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:10.533417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94708c56-0154-4ff4-a372-d7ff12132fd9 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:10.631141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94708c56-0154-4ff4-a372-d7ff12132fd9 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:10.647755 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:10.664424 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e32cd3-efff-4b54-af1f-5dbba49b1764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:10.665043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e32cd3-efff-4b54-af1f-5dbba49b1764 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:10.665320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e32cd3-efff-4b54-af1f-5dbba49b1764 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:10.665589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e32cd3-efff-4b54-af1f-5dbba49b1764 None admin] PATH_INFO: `/v3/users/8fe7ba2e765143709067649d7d3f7462` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:10.679407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e32cd3-efff-4b54-af1f-5dbba49b1764 None admin] RBAC: Authorizing `identity:get_user(user_id=8fe7ba2e765143709067649d7d3f7462)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:10.680831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e32cd3-efff-4b54-af1f-5dbba49b1764 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:10.682407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 560/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:39:10 2022] GET /identity/v3/users/8fe7ba2e765143709067649d7d3f7462 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:10.688541 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e9995-e4a5-4680-aa89-233bb3ee232b None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:10.782970 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e9995-e4a5-4680-aa89-233bb3ee232b None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:10.814257 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d6f42-4bda-4fd2-9e75-c9a455eb1d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:10.814882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d6f42-4bda-4fd2-9e75-c9a455eb1d30 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:10.815162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d6f42-4bda-4fd2-9e75-c9a455eb1d30 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:10.815435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d6f42-4bda-4fd2-9e75-c9a455eb1d30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:10.866404 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:10.890685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d6f42-4bda-4fd2-9e75-c9a455eb1d30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:10.892049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d6f42-4bda-4fd2-9e75-c9a455eb1d30 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:10.996630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 584/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:39:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:39:11.003215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e32cd3-efff-4b54-af1f-5dbba49b1764 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:11.103236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e32cd3-efff-4b54-af1f-5dbba49b1764 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:11.141112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21c16b-a12b-4a5d-943f-47a70023f135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:11.141668 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21c16b-a12b-4a5d-943f-47a70023f135 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:11.141947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21c16b-a12b-4a5d-943f-47a70023f135 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:11.142218 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21c16b-a12b-4a5d-943f-47a70023f135 None admin] PATH_INFO: `/v3/projects/26701951ea69458e8d37996f47a5b0a4` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:11.143361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21c16b-a12b-4a5d-943f-47a70023f135 None admin] RBAC: Authorizing `identity:get_project(project_id=26701951ea69458e8d37996f47a5b0a4)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:11.144780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21c16b-a12b-4a5d-943f-47a70023f135 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:11.147649 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 561/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:39:11 2022] GET /identity/v3/projects/26701951ea69458e8d37996f47a5b0a4 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:11.153834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d6f42-4bda-4fd2-9e75-c9a455eb1d30 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:11.252044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:11.271518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d6f42-4bda-4fd2-9e75-c9a455eb1d30 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:11.310946 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2baca43-a3d9-4001-bf25-1f98c89d987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:11.311806 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baca43-a3d9-4001-bf25-1f98c89d987e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:11.312077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baca43-a3d9-4001-bf25-1f98c89d987e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:11.312342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baca43-a3d9-4001-bf25-1f98c89d987e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:11.313498 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2baca43-a3d9-4001-bf25-1f98c89d987e None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=26701951ea69458e8d37996f47a5b0a4, user.id=8fe7ba2e765143709067649d7d3f7462) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:39:11.314020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2baca43-a3d9-4001-bf25-1f98c89d987e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:11.315598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2baca43-a3d9-4001-bf25-1f98c89d987e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:11.322128 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 585/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:39:11 2022] GET /identity/v3/role_assignments?user.id=8fe7ba2e765143709067649d7d3f7462&scope.project.id=26701951ea69458e8d37996f47a5b0a4&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:13.484184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a65c2b-57b1-4675-bf6c-a3142f82673d None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:13.484719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a65c2b-57b1-4675-bf6c-a3142f82673d None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:13.484719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a65c2b-57b1-4675-bf6c-a3142f82673d None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:13.486037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 562/1147] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:39:13.491861 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae69ca-0381-4f61-ac09-b6aea0938b1f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:13.492298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae69ca-0381-4f61-ac09-b6aea0938b1f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:13.492391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae69ca-0381-4f61-ac09-b6aea0938b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:13.510925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d6ae69ca-0381-4f61-ac09-b6aea0938b1f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:39:13.512097 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d6ae69ca-0381-4f61-ac09-b6aea0938b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:39:13.581009 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 586/1148] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:39:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:39:13.850533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a65c2b-57b1-4675-bf6c-a3142f82673d None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:13.895392 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:13.948093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a65c2b-57b1-4675-bf6c-a3142f82673d None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:13.980229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b816b57-8106-4386-bb37-c407a137416d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:13.980849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b816b57-8106-4386-bb37-c407a137416d None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:13.981130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b816b57-8106-4386-bb37-c407a137416d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:13.981400 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b816b57-8106-4386-bb37-c407a137416d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:13.991746 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b816b57-8106-4386-bb37-c407a137416d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-1b816b57-8106-4386-bb37-c407a137416d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:39:13.996416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:39:13.997509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:39:13.998596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:39:13.998596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:39:13.998596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:39:13.998596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:39:13.998596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:39:13.998596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:39:13.998596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:39:13.998596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:39:13.998596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:39:13.998596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:39:13.998596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:39:13.998596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:39:13.998596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 563/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:39:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:39:14.003485 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae69ca-0381-4f61-ac09-b6aea0938b1f None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:14.078375 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:14.100926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae69ca-0381-4f61-ac09-b6aea0938b1f None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:14.134849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9388e2-39fd-462e-bf60-833fc6be649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:14.135449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9388e2-39fd-462e-bf60-833fc6be649d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:14.135729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9388e2-39fd-462e-bf60-833fc6be649d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:14.136000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9388e2-39fd-462e-bf60-833fc6be649d None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:14.136689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9388e2-39fd-462e-bf60-833fc6be649d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:39:14.136989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9388e2-39fd-462e-bf60-833fc6be649d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:14.138370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9388e2-39fd-462e-bf60-833fc6be649d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:14.146834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 587/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:39:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:14.153080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b816b57-8106-4386-bb37-c407a137416d None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:14.249774 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b816b57-8106-4386-bb37-c407a137416d None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:14.252071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:14.283213 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5ef03-20f2-4c0c-8bf7-44d41b468722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:14.283823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5ef03-20f2-4c0c-8bf7-44d41b468722 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:14.284099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5ef03-20f2-4c0c-8bf7-44d41b468722 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:14.284368 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5ef03-20f2-4c0c-8bf7-44d41b468722 None admin] PATH_INFO: `/v3/users/8fe7ba2e765143709067649d7d3f7462` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:14.285449 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5ef03-20f2-4c0c-8bf7-44d41b468722 None admin] RBAC: Authorizing `identity:get_user(user_id=8fe7ba2e765143709067649d7d3f7462)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:14.286934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5ef03-20f2-4c0c-8bf7-44d41b468722 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:14.288457 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 564/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:39:14 2022] GET /identity/v3/users/8fe7ba2e765143709067649d7d3f7462 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:14.294219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9388e2-39fd-462e-bf60-833fc6be649d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:14.391580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9388e2-39fd-462e-bf60-833fc6be649d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:14.423976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a98b42-93a5-4be0-aef2-f6bce4c97cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:14.424589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a98b42-93a5-4be0-aef2-f6bce4c97cf2 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:14.424871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a98b42-93a5-4be0-aef2-f6bce4c97cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:14.425141 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a98b42-93a5-4be0-aef2-f6bce4c97cf2 None admin] PATH_INFO: `/v3/projects/26701951ea69458e8d37996f47a5b0a4` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:14.426240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a98b42-93a5-4be0-aef2-f6bce4c97cf2 None admin] RBAC: Authorizing `identity:get_project(project_id=26701951ea69458e8d37996f47a5b0a4)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:14.428214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a98b42-93a5-4be0-aef2-f6bce4c97cf2 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:14.429912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 588/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:39:14 2022] GET /identity/v3/projects/26701951ea69458e8d37996f47a5b0a4 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:14.435797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5ef03-20f2-4c0c-8bf7-44d41b468722 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:14.531413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5ef03-20f2-4c0c-8bf7-44d41b468722 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:14.569077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b29e6-850f-4250-821e-115dc68cde49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:14.569725 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b29e6-850f-4250-821e-115dc68cde49 None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:14.570008 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b29e6-850f-4250-821e-115dc68cde49 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:14.570282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b29e6-850f-4250-821e-115dc68cde49 None admin] PATH_INFO: `/v3/projects/26701951ea69458e8d37996f47a5b0a4/users/8fe7ba2e765143709067649d7d3f7462/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:14.572200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b29e6-850f-4250-821e-115dc68cde49 None admin] RBAC: Authorizing `identity:create_grant(project_id=26701951ea69458e8d37996f47a5b0a4, user_id=8fe7ba2e765143709067649d7d3f7462, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:14.573649 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b29e6-850f-4250-821e-115dc68cde49 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:14.680080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 565/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:39:14 2022] PUT /identity/v3/projects/26701951ea69458e8d37996f47a5b0a4/users/8fe7ba2e765143709067649d7d3f7462/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:39:16.864624 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27d1a9-32e3-4515-9c7a-975b1e475cda None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:16.865192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27d1a9-32e3-4515-9c7a-975b1e475cda None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:16.865192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27d1a9-32e3-4515-9c7a-975b1e475cda None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:16.867453 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 589/1154] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:39:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:39:16.872843 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f234f-baeb-4810-94e8-fe027714f77a None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:16.873273 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f234f-baeb-4810-94e8-fe027714f77a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:16.873386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f234f-baeb-4810-94e8-fe027714f77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:16.891162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-458f234f-baeb-4810-94e8-fe027714f77a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:39:16.892328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-458f234f-baeb-4810-94e8-fe027714f77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:39:16.917289 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:16.956575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 566/1155] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:39:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:39:17.221944 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27d1a9-32e3-4515-9c7a-975b1e475cda None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:17.231765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:17.323507 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27d1a9-32e3-4515-9c7a-975b1e475cda None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:17.356587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478b893-32e9-45a3-a876-a203e9997038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:17.357199 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478b893-32e9-45a3-a876-a203e9997038 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:17.357473 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478b893-32e9-45a3-a876-a203e9997038 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:17.357740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478b893-32e9-45a3-a876-a203e9997038 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:17.367900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478b893-32e9-45a3-a876-a203e9997038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-8478b893-32e9-45a3-a876-a203e9997038 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:39:17.372592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:39:17.373673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:39:17.374647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:39:17.374647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:39:17.374647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:39:17.374647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:39:17.374647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:39:17.374647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:39:17.374647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:39:17.374647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:39:17.374647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:39:17.374647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:39:17.374647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:39:17.374647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:39:17.374647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 590/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:39:17.379672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f234f-baeb-4810-94e8-fe027714f77a None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:17.476794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f234f-baeb-4810-94e8-fe027714f77a None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:17.509928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecff8b-608a-4759-97ed-6efa64d6b62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:17.510545 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecff8b-608a-4759-97ed-6efa64d6b62a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:17.510837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecff8b-608a-4759-97ed-6efa64d6b62a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:17.511108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecff8b-608a-4759-97ed-6efa64d6b62a None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:17.511761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecff8b-608a-4759-97ed-6efa64d6b62a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:39:17.512062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecff8b-608a-4759-97ed-6efa64d6b62a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:17.513413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecff8b-608a-4759-97ed-6efa64d6b62a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:17.521996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 567/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:39:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:17.527624 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478b893-32e9-45a3-a876-a203e9997038 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:17.589658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:17.622524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478b893-32e9-45a3-a876-a203e9997038 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:17.655063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf048b2-4885-4b91-ac26-ad121cc134a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:17.655606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf048b2-4885-4b91-ac26-ad121cc134a5 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:17.655878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf048b2-4885-4b91-ac26-ad121cc134a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:17.656146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf048b2-4885-4b91-ac26-ad121cc134a5 None admin] PATH_INFO: `/v3/users/8fe7ba2e765143709067649d7d3f7462` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:17.657232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf048b2-4885-4b91-ac26-ad121cc134a5 None admin] RBAC: Authorizing `identity:get_user(user_id=8fe7ba2e765143709067649d7d3f7462)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:17.658659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf048b2-4885-4b91-ac26-ad121cc134a5 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:17.660207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 591/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:39:17 2022] GET /identity/v3/users/8fe7ba2e765143709067649d7d3f7462 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:17.666134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecff8b-608a-4759-97ed-6efa64d6b62a None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:17.690947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:17.763419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecff8b-608a-4759-97ed-6efa64d6b62a None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:17.798800 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574ff58-fc42-4722-bc28-d69eb5afd46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:17.799405 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574ff58-fc42-4722-bc28-d69eb5afd46c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:17.799701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574ff58-fc42-4722-bc28-d69eb5afd46c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:17.799974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574ff58-fc42-4722-bc28-d69eb5afd46c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:17.865057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574ff58-fc42-4722-bc28-d69eb5afd46c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:17.866499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574ff58-fc42-4722-bc28-d69eb5afd46c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:17.889181 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:17.962320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 568/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:39:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:39:17.968201 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf048b2-4885-4b91-ac26-ad121cc134a5 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:18.062402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf048b2-4885-4b91-ac26-ad121cc134a5 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:18.094110 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6924b84-4633-426c-87bd-ba6ed60d16ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:18.094747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6924b84-4633-426c-87bd-ba6ed60d16ba None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:18.095021 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6924b84-4633-426c-87bd-ba6ed60d16ba None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:18.095289 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6924b84-4633-426c-87bd-ba6ed60d16ba None admin] PATH_INFO: `/v3/projects/26701951ea69458e8d37996f47a5b0a4` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:18.096375 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6924b84-4633-426c-87bd-ba6ed60d16ba None admin] RBAC: Authorizing `identity:get_project(project_id=26701951ea69458e8d37996f47a5b0a4)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:18.097769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6924b84-4633-426c-87bd-ba6ed60d16ba None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:18.099434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 592/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:39:17 2022] GET /identity/v3/projects/26701951ea69458e8d37996f47a5b0a4 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:18.105703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574ff58-fc42-4722-bc28-d69eb5afd46c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:18.200371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574ff58-fc42-4722-bc28-d69eb5afd46c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:18.222254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:18.232534 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f66ee7-8ef3-4eeb-b1d8-70548c3513ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:18.235962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f66ee7-8ef3-4eeb-b1d8-70548c3513ce None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:18.235962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f66ee7-8ef3-4eeb-b1d8-70548c3513ce None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:18.235962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f66ee7-8ef3-4eeb-b1d8-70548c3513ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:18.235962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f66ee7-8ef3-4eeb-b1d8-70548c3513ce None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=26701951ea69458e8d37996f47a5b0a4, user.id=8fe7ba2e765143709067649d7d3f7462) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:39:18.235962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f66ee7-8ef3-4eeb-b1d8-70548c3513ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:18.236470 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f66ee7-8ef3-4eeb-b1d8-70548c3513ce None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:18.242051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 569/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:39:18 2022] GET /identity/v3/role_assignments?user.id=8fe7ba2e765143709067649d7d3f7462&scope.project.id=26701951ea69458e8d37996f47a5b0a4&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:20.390358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76777615-e408-4b33-a5e9-c03db3d67977 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:20.390919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76777615-e408-4b33-a5e9-c03db3d67977 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:20.390919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76777615-e408-4b33-a5e9-c03db3d67977 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:20.392194 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 593/1162] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:39:20.397900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73296e4c-6339-4dce-bb00-0149bcfde4e3 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:20.398354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73296e4c-6339-4dce-bb00-0149bcfde4e3 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:20.398446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73296e4c-6339-4dce-bb00-0149bcfde4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:20.416623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-73296e4c-6339-4dce-bb00-0149bcfde4e3 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:39:20.417773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-73296e4c-6339-4dce-bb00-0149bcfde4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:39:20.481711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 570/1163] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:39:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:39:20.744627 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76777615-e408-4b33-a5e9-c03db3d67977 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:20.747490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:20.842712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76777615-e408-4b33-a5e9-c03db3d67977 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:20.873515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444ff63-fb1a-4812-be38-c1e60007ab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:20.874102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444ff63-fb1a-4812-be38-c1e60007ab0e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:20.874376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444ff63-fb1a-4812-be38-c1e60007ab0e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:20.874680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444ff63-fb1a-4812-be38-c1e60007ab0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:20.937955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444ff63-fb1a-4812-be38-c1e60007ab0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:20.939389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444ff63-fb1a-4812-be38-c1e60007ab0e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:20.941088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:21.032787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 594/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:39:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:39:21.039639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73296e4c-6339-4dce-bb00-0149bcfde4e3 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:21.136026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73296e4c-6339-4dce-bb00-0149bcfde4e3 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:21.154175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:21.171135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcd3fd-2c5e-4ea1-9037-95459be8c9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:21.171761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcd3fd-2c5e-4ea1-9037-95459be8c9a9 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:21.172042 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcd3fd-2c5e-4ea1-9037-95459be8c9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:21.172314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcd3fd-2c5e-4ea1-9037-95459be8c9a9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:21.178895 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcd3fd-2c5e-4ea1-9037-95459be8c9a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:21.180394 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcd3fd-2c5e-4ea1-9037-95459be8c9a9 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-15fcd3fd-2c5e-4ea1-9037-95459be8c9a9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:39:21.192170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:39:21.193473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:39:21.194395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 571/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 12:39:21 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:39:21.199377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444ff63-fb1a-4812-be38-c1e60007ab0e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:21.293910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444ff63-fb1a-4812-be38-c1e60007ab0e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:21.295340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:21.331439 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64033f7d-a2cc-47c9-bff4-ba8b4cd9f02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:21.332031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64033f7d-a2cc-47c9-bff4-ba8b4cd9f02b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:21.332310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64033f7d-a2cc-47c9-bff4-ba8b4cd9f02b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:21.332582 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64033f7d-a2cc-47c9-bff4-ba8b4cd9f02b None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:21.333222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64033f7d-a2cc-47c9-bff4-ba8b4cd9f02b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:39:21.334282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64033f7d-a2cc-47c9-bff4-ba8b4cd9f02b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:21.336391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64033f7d-a2cc-47c9-bff4-ba8b4cd9f02b None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:21.345136 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 595/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Fri Sep 9 12:39:21 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:23.471599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb4391-5816-444c-ae98-95184ae5f66c None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:23.472106 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb4391-5816-444c-ae98-95184ae5f66c None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:23.472222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb4391-5816-444c-ae98-95184ae5f66c None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:23.473497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 572/1167] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:39:23.478656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d95cc-ad12-4ad5-9ed3-4b488263d5d1 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:23.479088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d95cc-ad12-4ad5-9ed3-4b488263d5d1 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:23.479188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d95cc-ad12-4ad5-9ed3-4b488263d5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:23.497384 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-091d95cc-ad12-4ad5-9ed3-4b488263d5d1 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:39:23.498581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-091d95cc-ad12-4ad5-9ed3-4b488263d5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:39:23.561339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 596/1168] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:39:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:39:23.830336 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb4391-5816-444c-ae98-95184ae5f66c None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:23.928272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb4391-5816-444c-ae98-95184ae5f66c None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:23.960668 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd3a00-51df-4874-8153-b94aad75009e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:23.961264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd3a00-51df-4874-8153-b94aad75009e None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:23.961542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd3a00-51df-4874-8153-b94aad75009e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:23.961813 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd3a00-51df-4874-8153-b94aad75009e None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:23.962464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd3a00-51df-4874-8153-b94aad75009e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:23.963839 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd3a00-51df-4874-8153-b94aad75009e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:24.029137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 573/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Fri Sep 9 12:39:23 2022] POST /identity/v3/domains => generated 267 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:39:26.191420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4894ee-7876-46a7-bf6d-a0d7b75de7f8 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:26.191934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4894ee-7876-46a7-bf6d-a0d7b75de7f8 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:26.192028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4894ee-7876-46a7-bf6d-a0d7b75de7f8 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:26.193860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 597/1170] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:39:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:39:26.199001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb718f4-bfe0-4a37-994e-530526b63869 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:26.199389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb718f4-bfe0-4a37-994e-530526b63869 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:26.199546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb718f4-bfe0-4a37-994e-530526b63869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:26.217083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-2bb718f4-bfe0-4a37-994e-530526b63869 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:39:26.218240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2bb718f4-bfe0-4a37-994e-530526b63869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:39:26.246644 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:26.283534 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 574/1171] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:39:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:39:26.549169 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4894ee-7876-46a7-bf6d-a0d7b75de7f8 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:26.644294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:26.651882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4894ee-7876-46a7-bf6d-a0d7b75de7f8 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:26.684727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59729425-c837-4c4c-b29c-660de443167a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:26.685351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59729425-c837-4c4c-b29c-660de443167a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:26.685623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59729425-c837-4c4c-b29c-660de443167a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:26.685889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59729425-c837-4c4c-b29c-660de443167a None admin] PATH_INFO: `/v3/domains/06852c65cc6b4321a7e83709f71df794` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:26.696860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59729425-c837-4c4c-b29c-660de443167a None admin] RBAC: Authorizing `identity:get_domain(domain_id=06852c65cc6b4321a7e83709f71df794)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:26.699668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59729425-c837-4c4c-b29c-660de443167a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:26.699897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 598/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Fri Sep 9 12:39:26 2022] GET /identity/v3/domains/06852c65cc6b4321a7e83709f71df794 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:26.705720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb718f4-bfe0-4a37-994e-530526b63869 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:26.800495 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb718f4-bfe0-4a37-994e-530526b63869 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:26.832737 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde7901-1138-4f18-ab3b-f1c0ceee4ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:26.833965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde7901-1138-4f18-ab3b-f1c0ceee4ac1 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:26.834242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde7901-1138-4f18-ab3b-f1c0ceee4ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:26.834536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde7901-1138-4f18-ab3b-f1c0ceee4ac1 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:26.835184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde7901-1138-4f18-ab3b-f1c0ceee4ac1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:26.836590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde7901-1138-4f18-ab3b-f1c0ceee4ac1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:26.919779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 575/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:39:26 2022] POST /identity/v3/projects => generated 361 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:39:29.074800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86ec40-536b-460a-a123-95ce43290398 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:29.075365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86ec40-536b-460a-a123-95ce43290398 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:29.075365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86ec40-536b-460a-a123-95ce43290398 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:29.076678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 599/1174] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:39:29.081952 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83df3fd-1c8e-432b-9f90-1054f0174e36 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:29.082326 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83df3fd-1c8e-432b-9f90-1054f0174e36 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:29.082530 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83df3fd-1c8e-432b-9f90-1054f0174e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:29.100778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-c83df3fd-1c8e-432b-9f90-1054f0174e36 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:39:29.101947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c83df3fd-1c8e-432b-9f90-1054f0174e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:39:29.115903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:29.167146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 576/1175] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:39:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:39:29.443827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86ec40-536b-460a-a123-95ce43290398 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:29.541688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86ec40-536b-460a-a123-95ce43290398 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:29.573690 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304de2b-3bb0-4f43-b5f0-dabd2401f6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:29.574302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304de2b-3bb0-4f43-b5f0-dabd2401f6f9 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:29.574607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304de2b-3bb0-4f43-b5f0-dabd2401f6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:29.574881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304de2b-3bb0-4f43-b5f0-dabd2401f6f9 None admin] PATH_INFO: `/v3/domains/06852c65cc6b4321a7e83709f71df794` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:29.575976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304de2b-3bb0-4f43-b5f0-dabd2401f6f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=06852c65cc6b4321a7e83709f71df794)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:29.577378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304de2b-3bb0-4f43-b5f0-dabd2401f6f9 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:29.578944 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 600/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Fri Sep 9 12:39:29 2022] GET /identity/v3/domains/06852c65cc6b4321a7e83709f71df794 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:29.585350 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83df3fd-1c8e-432b-9f90-1054f0174e36 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:29.679513 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83df3fd-1c8e-432b-9f90-1054f0174e36 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:29.713646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1431f0-21e6-4f0c-8d09-876439bfa45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:29.714178 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1431f0-21e6-4f0c-8d09-876439bfa45d None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:29.714458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1431f0-21e6-4f0c-8d09-876439bfa45d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:29.714765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1431f0-21e6-4f0c-8d09-876439bfa45d None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:29.715397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1431f0-21e6-4f0c-8d09-876439bfa45d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:29.716809 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1431f0-21e6-4f0c-8d09-876439bfa45d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:29.809526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 577/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:39:29 2022] POST /identity/v3/users => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:39:31.965600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb48d37-a773-472d-8af5-7660f9ce50a8 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:31.966130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb48d37-a773-472d-8af5-7660f9ce50a8 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:31.966130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb48d37-a773-472d-8af5-7660f9ce50a8 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:31.967487 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 601/1178] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:39:31.972581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45140d91-9b42-4919-99f1-7e2956996e0f None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:31.972981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45140d91-9b42-4919-99f1-7e2956996e0f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:31.973138 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45140d91-9b42-4919-99f1-7e2956996e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:31.987205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:31.993177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-45140d91-9b42-4919-99f1-7e2956996e0f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:39:31.994351 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-45140d91-9b42-4919-99f1-7e2956996e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:39:32.057416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 578/1179] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:39:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:39:32.327012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb48d37-a773-472d-8af5-7660f9ce50a8 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:32.351608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:32.423877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb48d37-a773-472d-8af5-7660f9ce50a8 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:32.455896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500cc65-4326-450a-a242-2d5940cfa52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:32.456510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500cc65-4326-450a-a242-2d5940cfa52c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:32.456786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500cc65-4326-450a-a242-2d5940cfa52c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:32.457056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500cc65-4326-450a-a242-2d5940cfa52c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:32.467740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500cc65-4326-450a-a242-2d5940cfa52c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-2500cc65-4326-450a-a242-2d5940cfa52c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:39:32.472440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:39:32.473599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:39:32.475425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:39:32.475425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:39:32.475425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:39:32.475425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:39:32.475425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:39:32.475425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:39:32.475425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:39:32.475425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:39:32.475425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:39:32.475425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:39:32.475425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:39:32.475425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:39:32.475425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 602/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:39:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:39:32.480169 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45140d91-9b42-4919-99f1-7e2956996e0f None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:32.576854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45140d91-9b42-4919-99f1-7e2956996e0f None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:32.609561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32515738-6d23-4486-9106-a7491fe3225d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:32.610098 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32515738-6d23-4486-9106-a7491fe3225d None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:32.610380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32515738-6d23-4486-9106-a7491fe3225d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:32.610683 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32515738-6d23-4486-9106-a7491fe3225d None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:32.611336 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32515738-6d23-4486-9106-a7491fe3225d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:39:32.611635 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32515738-6d23-4486-9106-a7491fe3225d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:32.612985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32515738-6d23-4486-9106-a7491fe3225d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:32.615201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:32.621187 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 579/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:39:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:32.627052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500cc65-4326-450a-a242-2d5940cfa52c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:32.708958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:32.723427 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500cc65-4326-450a-a242-2d5940cfa52c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:32.754865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9aca4f-cf4e-4559-b6a4-d0e5a89336ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:32.755990 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9aca4f-cf4e-4559-b6a4-d0e5a89336ec None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:32.756257 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9aca4f-cf4e-4559-b6a4-d0e5a89336ec None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:32.756524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9aca4f-cf4e-4559-b6a4-d0e5a89336ec None admin] PATH_INFO: `/v3/users/b0294456114e453fa5823fbb329aac3a` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:32.769979 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9aca4f-cf4e-4559-b6a4-d0e5a89336ec None admin] RBAC: Authorizing `identity:get_user(user_id=b0294456114e453fa5823fbb329aac3a)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:32.776210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9aca4f-cf4e-4559-b6a4-d0e5a89336ec None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:32.776210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 603/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:39:32 2022] GET /identity/v3/users/b0294456114e453fa5823fbb329aac3a => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:32.779434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32515738-6d23-4486-9106-a7491fe3225d None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:32.875762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32515738-6d23-4486-9106-a7491fe3225d None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:32.908426 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe2bd1-77cd-43ab-823b-f8416823c9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:32.909026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe2bd1-77cd-43ab-823b-f8416823c9eb None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:32.909309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe2bd1-77cd-43ab-823b-f8416823c9eb None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:32.909663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe2bd1-77cd-43ab-823b-f8416823c9eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:32.971281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:32.975496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe2bd1-77cd-43ab-823b-f8416823c9eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:32.976921 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe2bd1-77cd-43ab-823b-f8416823c9eb None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:33.070079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 580/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:39:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:39:33.075992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9aca4f-cf4e-4559-b6a4-d0e5a89336ec None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:33.171276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9aca4f-cf4e-4559-b6a4-d0e5a89336ec None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:33.204448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287664bc-acce-4908-98cf-a569a86c8679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:33.205066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287664bc-acce-4908-98cf-a569a86c8679 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:33.205345 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287664bc-acce-4908-98cf-a569a86c8679 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:33.205618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287664bc-acce-4908-98cf-a569a86c8679 None admin] PATH_INFO: `/v3/projects/63004d64208d490da22c956f04bac447` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:33.206753 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287664bc-acce-4908-98cf-a569a86c8679 None admin] RBAC: Authorizing `identity:get_project(project_id=63004d64208d490da22c956f04bac447)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:33.208179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287664bc-acce-4908-98cf-a569a86c8679 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:33.210077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 604/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:39:33 2022] GET /identity/v3/projects/63004d64208d490da22c956f04bac447 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:33.216417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe2bd1-77cd-43ab-823b-f8416823c9eb None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:33.309620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:33.313575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe2bd1-77cd-43ab-823b-f8416823c9eb None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:33.350428 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877ebc0-5072-413a-ae1a-e011b054a9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:33.351057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877ebc0-5072-413a-ae1a-e011b054a9a2 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:33.351336 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877ebc0-5072-413a-ae1a-e011b054a9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:33.351608 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877ebc0-5072-413a-ae1a-e011b054a9a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:33.352318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877ebc0-5072-413a-ae1a-e011b054a9a2 None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=63004d64208d490da22c956f04bac447, user.id=b0294456114e453fa5823fbb329aac3a) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:39:33.352620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877ebc0-5072-413a-ae1a-e011b054a9a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:33.354029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877ebc0-5072-413a-ae1a-e011b054a9a2 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:33.359351 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 581/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:39:33 2022] GET /identity/v3/role_assignments?user.id=b0294456114e453fa5823fbb329aac3a&scope.project.id=63004d64208d490da22c956f04bac447&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:35.489637 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e688a8-967b-4ff0-a198-306691474d00 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:35.490205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e688a8-967b-4ff0-a198-306691474d00 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:35.490205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e688a8-967b-4ff0-a198-306691474d00 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:35.491555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 605/1186] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:39:35.496848 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4fce7-f838-4441-a418-e5ce0220574e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:35.497265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4fce7-f838-4441-a418-e5ce0220574e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:35.497396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4fce7-f838-4441-a418-e5ce0220574e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:35.515504 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-39a4fce7-f838-4441-a418-e5ce0220574e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:39:35.516662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-39a4fce7-f838-4441-a418-e5ce0220574e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:39:35.584442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 582/1187] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:39:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:39:35.867783 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e688a8-967b-4ff0-a198-306691474d00 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:35.879726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:35.968407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e688a8-967b-4ff0-a198-306691474d00 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:36.000448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60620171-e4a1-4e8c-a769-a048950bf603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:36.001062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60620171-e4a1-4e8c-a769-a048950bf603 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:36.001340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60620171-e4a1-4e8c-a769-a048950bf603 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:36.001609 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60620171-e4a1-4e8c-a769-a048950bf603 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:36.011864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60620171-e4a1-4e8c-a769-a048950bf603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-60620171-e4a1-4e8c-a769-a048950bf603 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:39:36.016554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:39:36.017647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:39:36.018886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:39:36.018886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:39:36.018886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:39:36.018886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:39:36.018886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:39:36.018886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:39:36.018886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:39:36.018886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:39:36.018886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:39:36.018886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:39:36.018886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:39:36.018886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:39:36.018886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 606/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:39:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:39:36.023690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4fce7-f838-4441-a418-e5ce0220574e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:36.111469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:36.121643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4fce7-f838-4441-a418-e5ce0220574e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:36.153311 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c3a6b-2a39-43ea-bb44-49ee3b30e1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:36.153904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c3a6b-2a39-43ea-bb44-49ee3b30e1ca None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:36.154179 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c3a6b-2a39-43ea-bb44-49ee3b30e1ca None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:36.154447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c3a6b-2a39-43ea-bb44-49ee3b30e1ca None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:36.155125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c3a6b-2a39-43ea-bb44-49ee3b30e1ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:39:36.155420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c3a6b-2a39-43ea-bb44-49ee3b30e1ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:36.156771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c3a6b-2a39-43ea-bb44-49ee3b30e1ca None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:36.165056 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 583/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:39:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:36.170922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60620171-e4a1-4e8c-a769-a048950bf603 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:36.237903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:36.266603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60620171-e4a1-4e8c-a769-a048950bf603 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:36.299330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aab739-bf2d-44db-85b6-3d1f2d53a8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:36.299933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aab739-bf2d-44db-85b6-3d1f2d53a8af None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:36.300205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aab739-bf2d-44db-85b6-3d1f2d53a8af None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:36.300471 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aab739-bf2d-44db-85b6-3d1f2d53a8af None admin] PATH_INFO: `/v3/users/b0294456114e453fa5823fbb329aac3a` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:36.301559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aab739-bf2d-44db-85b6-3d1f2d53a8af None admin] RBAC: Authorizing `identity:get_user(user_id=b0294456114e453fa5823fbb329aac3a)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:36.303101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aab739-bf2d-44db-85b6-3d1f2d53a8af None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:36.304638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 607/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:39:36 2022] GET /identity/v3/users/b0294456114e453fa5823fbb329aac3a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:36.310399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c3a6b-2a39-43ea-bb44-49ee3b30e1ca None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:36.409577 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c3a6b-2a39-43ea-bb44-49ee3b30e1ca None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:36.443227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d72b7-71ac-47f0-b136-1968c1daa6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:36.444001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d72b7-71ac-47f0-b136-1968c1daa6ed None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:36.444283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d72b7-71ac-47f0-b136-1968c1daa6ed None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:36.444554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d72b7-71ac-47f0-b136-1968c1daa6ed None admin] PATH_INFO: `/v3/projects/63004d64208d490da22c956f04bac447` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:36.445656 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d72b7-71ac-47f0-b136-1968c1daa6ed None admin] RBAC: Authorizing `identity:get_project(project_id=63004d64208d490da22c956f04bac447)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:36.447103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d72b7-71ac-47f0-b136-1968c1daa6ed None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:36.448690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 584/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:39:36 2022] GET /identity/v3/projects/63004d64208d490da22c956f04bac447 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:36.454509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aab739-bf2d-44db-85b6-3d1f2d53a8af None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:36.548224 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aab739-bf2d-44db-85b6-3d1f2d53a8af None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:36.580364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06967e1e-dbba-48c8-8624-b8361b31b1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:36.580996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06967e1e-dbba-48c8-8624-b8361b31b1c0 None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:36.581270 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06967e1e-dbba-48c8-8624-b8361b31b1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:36.581537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06967e1e-dbba-48c8-8624-b8361b31b1c0 None admin] PATH_INFO: `/v3/projects/63004d64208d490da22c956f04bac447/users/b0294456114e453fa5823fbb329aac3a/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:36.584136 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06967e1e-dbba-48c8-8624-b8361b31b1c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=63004d64208d490da22c956f04bac447, user_id=b0294456114e453fa5823fbb329aac3a, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:36.585532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06967e1e-dbba-48c8-8624-b8361b31b1c0 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:36.661459 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:36.666687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 608/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:39:36 2022] PUT /identity/v3/projects/63004d64208d490da22c956f04bac447/users/b0294456114e453fa5823fbb329aac3a/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:39:38.847275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec839fee-c223-478a-8df0-6ebb98b79360 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:38.847806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec839fee-c223-478a-8df0-6ebb98b79360 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:38.847806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec839fee-c223-478a-8df0-6ebb98b79360 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:38.849687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 585/1193] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:39:38.854879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c7ef7-c675-482b-bb44-72a6e8838db8 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:38.855289 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c7ef7-c675-482b-bb44-72a6e8838db8 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:38.855419 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c7ef7-c675-482b-bb44-72a6e8838db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:38.875016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-e27c7ef7-c675-482b-bb44-72a6e8838db8 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:39:38.876191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e27c7ef7-c675-482b-bb44-72a6e8838db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:39:38.941246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 609/1194] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:39:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:39:39.217727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec839fee-c223-478a-8df0-6ebb98b79360 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:39.242335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:39.316950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec839fee-c223-478a-8df0-6ebb98b79360 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:39.349544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff11c3c-eb41-4fac-9259-1b6c60e756a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:39.350058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff11c3c-eb41-4fac-9259-1b6c60e756a3 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:39.350340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff11c3c-eb41-4fac-9259-1b6c60e756a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:39.350645 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff11c3c-eb41-4fac-9259-1b6c60e756a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:39.360804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff11c3c-eb41-4fac-9259-1b6c60e756a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-0ff11c3c-eb41-4fac-9259-1b6c60e756a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:39:39.366164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:39:39.367307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:39:39.368354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:39:39.368354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:39:39.368354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:39:39.368354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:39:39.368354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:39:39.368354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:39:39.368354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:39:39.368354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:39:39.368354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:39:39.368354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:39:39.368354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:39:39.368354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:39:39.368354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 586/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:39:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:39:39.373392 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c7ef7-c675-482b-bb44-72a6e8838db8 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:39.476252 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c7ef7-c675-482b-bb44-72a6e8838db8 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:39.502170 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:39.509995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f1223-c2ee-4e6f-ab9d-12f0a7bab05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:39.512685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f1223-c2ee-4e6f-ab9d-12f0a7bab05a None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:39.512685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f1223-c2ee-4e6f-ab9d-12f0a7bab05a None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:39.512685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f1223-c2ee-4e6f-ab9d-12f0a7bab05a None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:39.512685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f1223-c2ee-4e6f-ab9d-12f0a7bab05a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:39:39.512685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f1223-c2ee-4e6f-ab9d-12f0a7bab05a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:39.513553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f1223-c2ee-4e6f-ab9d-12f0a7bab05a None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:39.521720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 610/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:39:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:39.527924 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff11c3c-eb41-4fac-9259-1b6c60e756a3 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:39.606701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:39.624241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff11c3c-eb41-4fac-9259-1b6c60e756a3 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:39.656770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45ba42-c4ad-4f9b-befa-22d4bd7a7e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:39.657379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45ba42-c4ad-4f9b-befa-22d4bd7a7e96 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:39.657661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45ba42-c4ad-4f9b-befa-22d4bd7a7e96 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:39.657926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45ba42-c4ad-4f9b-befa-22d4bd7a7e96 None admin] PATH_INFO: `/v3/users/b0294456114e453fa5823fbb329aac3a` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:39.659036 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45ba42-c4ad-4f9b-befa-22d4bd7a7e96 None admin] RBAC: Authorizing `identity:get_user(user_id=b0294456114e453fa5823fbb329aac3a)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:39.660436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45ba42-c4ad-4f9b-befa-22d4bd7a7e96 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:39.662593 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 587/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Fri Sep 9 12:39:39 2022] GET /identity/v3/users/b0294456114e453fa5823fbb329aac3a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:39.668622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f1223-c2ee-4e6f-ab9d-12f0a7bab05a None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:39.767428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f1223-c2ee-4e6f-ab9d-12f0a7bab05a None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:39.800902 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea07fbd-d048-4afa-a007-8513f24a07d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:39.801495 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea07fbd-d048-4afa-a007-8513f24a07d1 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:39.802066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea07fbd-d048-4afa-a007-8513f24a07d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:39.802343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea07fbd-d048-4afa-a007-8513f24a07d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:39.857687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:39.867585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea07fbd-d048-4afa-a007-8513f24a07d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:39.884500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea07fbd-d048-4afa-a007-8513f24a07d1 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:39.963988 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 611/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:39:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:39:39.970027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45ba42-c4ad-4f9b-befa-22d4bd7a7e96 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:40.068881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45ba42-c4ad-4f9b-befa-22d4bd7a7e96 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:40.100829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b2a33-7bd9-4310-ab30-66c508d9786e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:40.101443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b2a33-7bd9-4310-ab30-66c508d9786e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:40.101720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b2a33-7bd9-4310-ab30-66c508d9786e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:40.101987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b2a33-7bd9-4310-ab30-66c508d9786e None admin] PATH_INFO: `/v3/projects/63004d64208d490da22c956f04bac447` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:40.103081 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b2a33-7bd9-4310-ab30-66c508d9786e None admin] RBAC: Authorizing `identity:get_project(project_id=63004d64208d490da22c956f04bac447)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:40.104486 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b2a33-7bd9-4310-ab30-66c508d9786e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:40.106082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 588/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Fri Sep 9 12:39:39 2022] GET /identity/v3/projects/63004d64208d490da22c956f04bac447 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:40.112120 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea07fbd-d048-4afa-a007-8513f24a07d1 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:39:40.200975 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:40.210735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea07fbd-d048-4afa-a007-8513f24a07d1 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:39:40.244697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d76e97-9fad-4dcd-87fb-02bf517b0034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:39:40.245286 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d76e97-9fad-4dcd-87fb-02bf517b0034 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:39:40.245562 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d76e97-9fad-4dcd-87fb-02bf517b0034 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:39:40.245832 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d76e97-9fad-4dcd-87fb-02bf517b0034 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:39:40.246581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d76e97-9fad-4dcd-87fb-02bf517b0034 None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=63004d64208d490da22c956f04bac447, user.id=b0294456114e453fa5823fbb329aac3a) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:39:40.246889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d76e97-9fad-4dcd-87fb-02bf517b0034 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:39:40.248301 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d76e97-9fad-4dcd-87fb-02bf517b0034 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:39:40.253787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 612/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:39:40 2022] GET /identity/v3/role_assignments?user.id=b0294456114e453fa5823fbb329aac3a&scope.project.id=63004d64208d490da22c956f04bac447&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:39:50.096244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=91 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:39:50.251290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=23 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:40:52.748247 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d464a-b04b-4f2d-b999-b233a0f4fe4f None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:52.748852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d464a-b04b-4f2d-b999-b233a0f4fe4f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:52.748852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d464a-b04b-4f2d-b999-b233a0f4fe4f None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:52.750129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 589/1201] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:40:52.755292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc2136-b480-402b-b355-03c48c6bebfd None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:52.755738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc2136-b480-402b-b355-03c48c6bebfd None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:52.755836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc2136-b480-402b-b355-03c48c6bebfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:52.774058 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-8ffc2136-b480-402b-b355-03c48c6bebfd None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:40:52.775284 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8ffc2136-b480-402b-b355-03c48c6bebfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:40:52.839607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 613/1202] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:40:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:40:53.116394 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d464a-b04b-4f2d-b999-b233a0f4fe4f None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:53.219326 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d464a-b04b-4f2d-b999-b233a0f4fe4f None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:53.251143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329088c-a948-449e-a710-1ac30656e7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:53.251749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329088c-a948-449e-a710-1ac30656e7a3 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:53.252067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329088c-a948-449e-a710-1ac30656e7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:53.252340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329088c-a948-449e-a710-1ac30656e7a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:53.263415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329088c-a948-449e-a710-1ac30656e7a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:53.264845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329088c-a948-449e-a710-1ac30656e7a3 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-f329088c-a948-449e-a710-1ac30656e7a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:40:53.279215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:40:53.280411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:40:53.281619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:40:53.281619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:40:53.281619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:40:53.281619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 590/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:40:53.286475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc2136-b480-402b-b355-03c48c6bebfd None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:53.383094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc2136-b480-402b-b355-03c48c6bebfd None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:53.412679 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:40:53.418947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610f431-8ca1-401a-9348-736675cec65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:53.419547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610f431-8ca1-401a-9348-736675cec65e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:53.419824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610f431-8ca1-401a-9348-736675cec65e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:53.420092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610f431-8ca1-401a-9348-736675cec65e None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:53.420729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610f431-8ca1-401a-9348-736675cec65e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:40:53.421027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610f431-8ca1-401a-9348-736675cec65e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:53.422367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610f431-8ca1-401a-9348-736675cec65e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:53.433218 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 614/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:40:53.439202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329088c-a948-449e-a710-1ac30656e7a3 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:53.491628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:40:53.537475 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329088c-a948-449e-a710-1ac30656e7a3 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:53.570591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd21a4f-678c-43ff-a7ef-2793020136c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:53.571189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd21a4f-678c-43ff-a7ef-2793020136c6 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:53.571469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd21a4f-678c-43ff-a7ef-2793020136c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:53.571767 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd21a4f-678c-43ff-a7ef-2793020136c6 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:53.572458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd21a4f-678c-43ff-a7ef-2793020136c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:53.573904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd21a4f-678c-43ff-a7ef-2793020136c6 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:53.664706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 591/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Fri Sep 9 12:40:53 2022] POST /identity/v3/users => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:40:55.824157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4c23c-d54f-4a5b-b85e-8b5730927337 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:55.824686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4c23c-d54f-4a5b-b85e-8b5730927337 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:55.824797 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4c23c-d54f-4a5b-b85e-8b5730927337 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:55.826125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 615/1206] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:40:55.831595 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e837e0b-75f6-4e9c-bef5-f6d1764bee4c None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:55.832080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e837e0b-75f6-4e9c-bef5-f6d1764bee4c None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:55.832175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e837e0b-75f6-4e9c-bef5-f6d1764bee4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:55.849897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-9e837e0b-75f6-4e9c-bef5-f6d1764bee4c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:40:55.851199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9e837e0b-75f6-4e9c-bef5-f6d1764bee4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:40:55.917550 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 592/1207] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:40:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:40:56.189124 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4c23c-d54f-4a5b-b85e-8b5730927337 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:56.290030 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4c23c-d54f-4a5b-b85e-8b5730927337 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:56.323897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddf77a-b48d-4b54-b919-820376cbf381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:56.324517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddf77a-b48d-4b54-b919-820376cbf381 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:56.324793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddf77a-b48d-4b54-b919-820376cbf381 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:56.325067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddf77a-b48d-4b54-b919-820376cbf381 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:56.335502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddf77a-b48d-4b54-b919-820376cbf381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-01ddf77a-b48d-4b54-b919-820376cbf381 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:40:56.341020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:40:56.342157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:40:56.343260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:40:56.343260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:40:56.343260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:40:56.343260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:40:56.343260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:40:56.343260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:40:56.343260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:40:56.343260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:40:56.343260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:40:56.343260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:40:56.343260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:40:56.343260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:40:56.343260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 616/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:40:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:40:56.348142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e837e0b-75f6-4e9c-bef5-f6d1764bee4c None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:56.388432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:40:56.445397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e837e0b-75f6-4e9c-bef5-f6d1764bee4c None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:56.477796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f668bd-fdc4-492f-aa88-2d75cb0d59eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:56.479313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f668bd-fdc4-492f-aa88-2d75cb0d59eb None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:56.479950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f668bd-fdc4-492f-aa88-2d75cb0d59eb None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:56.480290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f668bd-fdc4-492f-aa88-2d75cb0d59eb None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:56.480982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f668bd-fdc4-492f-aa88-2d75cb0d59eb None admin] RBAC: Adding query filter params (name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:40:56.481277 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f668bd-fdc4-492f-aa88-2d75cb0d59eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:56.482661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f668bd-fdc4-492f-aa88-2d75cb0d59eb None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:56.490417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 593/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:40:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:40:56.496783 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddf77a-b48d-4b54-b919-820376cbf381 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:56.547212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:40:56.594754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddf77a-b48d-4b54-b919-820376cbf381 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:56.626657 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea371b57-883f-40f4-9d87-80359ce20998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:56.627271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea371b57-883f-40f4-9d87-80359ce20998 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:56.627543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea371b57-883f-40f4-9d87-80359ce20998 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:56.627810 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea371b57-883f-40f4-9d87-80359ce20998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:56.638960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea371b57-883f-40f4-9d87-80359ce20998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:56.640396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea371b57-883f-40f4-9d87-80359ce20998 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-ea371b57-883f-40f4-9d87-80359ce20998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:40:56.655277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:40:56.656390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:40:56.657409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:40:56.657409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:40:56.657409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:40:56.657409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 617/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:40:56.662394 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f668bd-fdc4-492f-aa88-2d75cb0d59eb None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:56.758632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:40:56.762053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f668bd-fdc4-492f-aa88-2d75cb0d59eb None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:56.794372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3273ff-0525-4222-9c66-3e72f8540ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:56.794999 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3273ff-0525-4222-9c66-3e72f8540ca1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:56.795282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3273ff-0525-4222-9c66-3e72f8540ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:56.795558 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3273ff-0525-4222-9c66-3e72f8540ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:56.796192 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3273ff-0525-4222-9c66-3e72f8540ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:40:56.796494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3273ff-0525-4222-9c66-3e72f8540ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:56.797855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3273ff-0525-4222-9c66-3e72f8540ca1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:56.809495 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 594/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:40:56.815420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea371b57-883f-40f4-9d87-80359ce20998 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:56.912530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea371b57-883f-40f4-9d87-80359ce20998 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:56.913927 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:40:56.944515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f6af6-5bb2-4503-acf0-64d3786e35f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:56.945129 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f6af6-5bb2-4503-acf0-64d3786e35f7 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:56.945417 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f6af6-5bb2-4503-acf0-64d3786e35f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:56.949013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f6af6-5bb2-4503-acf0-64d3786e35f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:56.953234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f6af6-5bb2-4503-acf0-64d3786e35f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:56.954666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f6af6-5bb2-4503-acf0-64d3786e35f7 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-826f6af6-5bb2-4503-acf0-64d3786e35f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:40:56.966064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:40:56.967226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:40:56.968442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:40:56.968442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:40:56.968442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:40:56.968442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:40:56.968442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:40:56.968442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:40:56.968442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:40:56.968442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:40:56.968442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:40:56.968442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:40:56.968442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 618/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:40:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:40:56.973090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3273ff-0525-4222-9c66-3e72f8540ca1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:57.071079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3273ff-0525-4222-9c66-3e72f8540ca1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:57.104972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cb5a6-1113-449a-af7a-b4fb1c6326c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:57.105574 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cb5a6-1113-449a-af7a-b4fb1c6326c0 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:57.105856 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cb5a6-1113-449a-af7a-b4fb1c6326c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:57.106131 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cb5a6-1113-449a-af7a-b4fb1c6326c0 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:57.106897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cb5a6-1113-449a-af7a-b4fb1c6326c0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:40:57.107204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cb5a6-1113-449a-af7a-b4fb1c6326c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:57.108619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cb5a6-1113-449a-af7a-b4fb1c6326c0 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:57.127626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 595/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Fri Sep 9 12:40:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:40:57.133362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f6af6-5bb2-4503-acf0-64d3786e35f7 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:57.231111 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f6af6-5bb2-4503-acf0-64d3786e35f7 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:57.264203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc263fe5-6d2b-48eb-bf0e-8ca29fa90aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:57.264803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc263fe5-6d2b-48eb-bf0e-8ca29fa90aea None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:57.265080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc263fe5-6d2b-48eb-bf0e-8ca29fa90aea None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:57.265349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc263fe5-6d2b-48eb-bf0e-8ca29fa90aea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:57.296128 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:40:57.332568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc263fe5-6d2b-48eb-bf0e-8ca29fa90aea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:57.334047 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc263fe5-6d2b-48eb-bf0e-8ca29fa90aea None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:57.430007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 619/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:40:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:40:57.436466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cb5a6-1113-449a-af7a-b4fb1c6326c0 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:57.456053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:40:57.535946 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cb5a6-1113-449a-af7a-b4fb1c6326c0 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:57.569118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b16fbf-682b-4281-931e-33e8f9cc9193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:57.569741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b16fbf-682b-4281-931e-33e8f9cc9193 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:57.570023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b16fbf-682b-4281-931e-33e8f9cc9193 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:57.570300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b16fbf-682b-4281-931e-33e8f9cc9193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:57.580687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b16fbf-682b-4281-931e-33e8f9cc9193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:57.582116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b16fbf-682b-4281-931e-33e8f9cc9193 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-f4b16fbf-682b-4281-931e-33e8f9cc9193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:40:57.596501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:40:57.597640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:40:57.598605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:40:57.598605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:40:57.598605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:40:57.598605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 596/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:40:57.603987 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc263fe5-6d2b-48eb-bf0e-8ca29fa90aea None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:57.662064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:40:57.699250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc263fe5-6d2b-48eb-bf0e-8ca29fa90aea None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:57.731162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee629dbd-d416-4b0c-a8f0-8b424cf1b55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:57.731754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee629dbd-d416-4b0c-a8f0-8b424cf1b55c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:57.732029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee629dbd-d416-4b0c-a8f0-8b424cf1b55c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:57.732311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee629dbd-d416-4b0c-a8f0-8b424cf1b55c None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:57.733216 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee629dbd-d416-4b0c-a8f0-8b424cf1b55c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:40:57.733693 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee629dbd-d416-4b0c-a8f0-8b424cf1b55c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:57.735599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee629dbd-d416-4b0c-a8f0-8b424cf1b55c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:57.747105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 620/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:40:57.752933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b16fbf-682b-4281-931e-33e8f9cc9193 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:57.825236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:40:57.854390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b16fbf-682b-4281-931e-33e8f9cc9193 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:57.887575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba78547-0200-4636-822c-bb44f0db2db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:57.888236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba78547-0200-4636-822c-bb44f0db2db1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:57.888512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba78547-0200-4636-822c-bb44f0db2db1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:57.888783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba78547-0200-4636-822c-bb44f0db2db1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:57.895288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba78547-0200-4636-822c-bb44f0db2db1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:57.896714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba78547-0200-4636-822c-bb44f0db2db1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-1ba78547-0200-4636-822c-bb44f0db2db1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:40:57.908278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:40:57.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:40:57.910551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:40:57.910551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:40:57.910551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:40:57.910551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:40:57.910551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:40:57.910551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:40:57.910551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:40:57.910551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:40:57.910551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 597/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:40:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:40:57.915865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee629dbd-d416-4b0c-a8f0-8b424cf1b55c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:58.017288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee629dbd-d416-4b0c-a8f0-8b424cf1b55c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:58.038633 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:40:58.050225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542097a-40a8-4b07-9560-f6b48840aaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:58.050865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542097a-40a8-4b07-9560-f6b48840aaef None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:58.051143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542097a-40a8-4b07-9560-f6b48840aaef None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:58.051412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542097a-40a8-4b07-9560-f6b48840aaef None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:58.052131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542097a-40a8-4b07-9560-f6b48840aaef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:40:58.052433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542097a-40a8-4b07-9560-f6b48840aaef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:58.053840 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542097a-40a8-4b07-9560-f6b48840aaef None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:58.066133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 621/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:40:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:40:58.073710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba78547-0200-4636-822c-bb44f0db2db1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:40:58.176389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba78547-0200-4636-822c-bb44f0db2db1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:40:58.205046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:40:58.210674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fdddb-5ce0-43a5-b0d0-d471dc506beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:40:58.211267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fdddb-5ce0-43a5-b0d0-d471dc506beb None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:40:58.211542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fdddb-5ce0-43a5-b0d0-d471dc506beb None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:40:58.211845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fdddb-5ce0-43a5-b0d0-d471dc506beb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:40:58.212778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fdddb-5ce0-43a5-b0d0-d471dc506beb None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=33e177d86450489cacc86578ba26509d) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:40:58.213087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fdddb-5ce0-43a5-b0d0-d471dc506beb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:40:58.214552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fdddb-5ce0-43a5-b0d0-d471dc506beb None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:40:58.219939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 598/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:40:58 2022] GET /identity/v3/role_assignments?user.id=33e177d86450489cacc86578ba26509d&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:00.397626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aab655-9ac0-4273-af4f-1a731f2f138b None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:00.398140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aab655-9ac0-4273-af4f-1a731f2f138b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:00.398140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aab655-9ac0-4273-af4f-1a731f2f138b None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:00.399456 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 622/1220] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:41:00.405382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497ee74-46d0-401e-88fc-ddcb11725fa4 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:00.405812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497ee74-46d0-401e-88fc-ddcb11725fa4 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:00.405954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497ee74-46d0-401e-88fc-ddcb11725fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:00.424808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-7497ee74-46d0-401e-88fc-ddcb11725fa4 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:41:00.425982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7497ee74-46d0-401e-88fc-ddcb11725fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:41:00.493267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 599/1221] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:41:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:41:00.764390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aab655-9ac0-4273-af4f-1a731f2f138b None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:00.866090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aab655-9ac0-4273-af4f-1a731f2f138b None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:00.899156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce9ac9-d65d-45f6-ad4c-864e91a4531f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:00.900438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce9ac9-d65d-45f6-ad4c-864e91a4531f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:00.900722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce9ac9-d65d-45f6-ad4c-864e91a4531f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:00.900989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce9ac9-d65d-45f6-ad4c-864e91a4531f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:00.910760 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce9ac9-d65d-45f6-ad4c-864e91a4531f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-8bce9ac9-d65d-45f6-ad4c-864e91a4531f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:41:00.915489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:00.918553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:00.921632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:00.921632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:00.921632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:00.921632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:00.921632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:41:00.921632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:41:00.921632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:41:00.921632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:41:00.921632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:41:00.921632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:41:00.921632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:41:00.921632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:00.921632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 623/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:41:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:00.922965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497ee74-46d0-401e-88fc-ddcb11725fa4 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:01.019820 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497ee74-46d0-401e-88fc-ddcb11725fa4 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:01.041960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:01.052150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e15071-3c8c-4d10-a804-0bda41206f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:01.053692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e15071-3c8c-4d10-a804-0bda41206f29 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:01.053692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e15071-3c8c-4d10-a804-0bda41206f29 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:01.053692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e15071-3c8c-4d10-a804-0bda41206f29 None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:01.054043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e15071-3c8c-4d10-a804-0bda41206f29 None admin] RBAC: Adding query filter params (name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:01.054339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e15071-3c8c-4d10-a804-0bda41206f29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:01.055762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e15071-3c8c-4d10-a804-0bda41206f29 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:01.064337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 600/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:41:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:41:01.070086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce9ac9-d65d-45f6-ad4c-864e91a4531f None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:01.113489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:01.167459 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce9ac9-d65d-45f6-ad4c-864e91a4531f None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:01.202284 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7c0ff-a17d-47fc-a4a1-1eb47762adb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:01.202954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7c0ff-a17d-47fc-a4a1-1eb47762adb8 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:01.203229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7c0ff-a17d-47fc-a4a1-1eb47762adb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:01.203499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7c0ff-a17d-47fc-a4a1-1eb47762adb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:01.213895 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7c0ff-a17d-47fc-a4a1-1eb47762adb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:01.215341 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7c0ff-a17d-47fc-a4a1-1eb47762adb8 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-05f7c0ff-a17d-47fc-a4a1-1eb47762adb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:41:01.229876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:01.231735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:41:01.232814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:41:01.232814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:01.232814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:01.232814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 624/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:01.236787 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e15071-3c8c-4d10-a804-0bda41206f29 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:01.338020 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e15071-3c8c-4d10-a804-0bda41206f29 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:01.371900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63578ef2-a925-4714-9282-e03383aee5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:01.372509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63578ef2-a925-4714-9282-e03383aee5fd None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:01.372791 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63578ef2-a925-4714-9282-e03383aee5fd None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:01.373070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63578ef2-a925-4714-9282-e03383aee5fd None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:01.373758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63578ef2-a925-4714-9282-e03383aee5fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:01.374100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63578ef2-a925-4714-9282-e03383aee5fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:01.375494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63578ef2-a925-4714-9282-e03383aee5fd None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:01.387267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 601/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:01.393000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7c0ff-a17d-47fc-a4a1-1eb47762adb8 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:01.485925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:01.492293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7c0ff-a17d-47fc-a4a1-1eb47762adb8 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:01.524710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cb722-9b77-42b1-b6f5-5ffc246973fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:01.525593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cb722-9b77-42b1-b6f5-5ffc246973fb None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:01.525869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cb722-9b77-42b1-b6f5-5ffc246973fb None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:01.526135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cb722-9b77-42b1-b6f5-5ffc246973fb None admin] PATH_INFO: `/v3/users/placement` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:01.533581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cb722-9b77-42b1-b6f5-5ffc246973fb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:01.535019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cb722-9b77-42b1-b6f5-5ffc246973fb None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-614cb722-9b77-42b1-b6f5-5ffc246973fb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:41:01.546386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:01.551101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:01.552294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:01.552294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:01.552294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:41:01.552294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:41:01.552294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:41:01.552294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:41:01.552294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:41:01.552294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:41:01.552294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:41:01.552294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:01.552294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 625/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:01.553389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63578ef2-a925-4714-9282-e03383aee5fd None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:01.599512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:01.650248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63578ef2-a925-4714-9282-e03383aee5fd None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:01.683957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff9725-ef1c-4e0c-859a-c0fe223e94d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:01.684553 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff9725-ef1c-4e0c-859a-c0fe223e94d1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:01.684831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff9725-ef1c-4e0c-859a-c0fe223e94d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:01.685103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff9725-ef1c-4e0c-859a-c0fe223e94d1 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:01.685819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff9725-ef1c-4e0c-859a-c0fe223e94d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:01.686120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff9725-ef1c-4e0c-859a-c0fe223e94d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:01.687823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff9725-ef1c-4e0c-859a-c0fe223e94d1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:01.704295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 602/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Fri Sep 9 12:41:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:01.710100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cb722-9b77-42b1-b6f5-5ffc246973fb None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:01.806861 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cb722-9b77-42b1-b6f5-5ffc246973fb None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:01.840660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199993d5-dcd7-43c9-9872-8f4837562e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:01.844884 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199993d5-dcd7-43c9-9872-8f4837562e51 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:01.844884 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199993d5-dcd7-43c9-9872-8f4837562e51 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:01.844884 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199993d5-dcd7-43c9-9872-8f4837562e51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:01.853961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199993d5-dcd7-43c9-9872-8f4837562e51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:01.854858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199993d5-dcd7-43c9-9872-8f4837562e51 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-199993d5-dcd7-43c9-9872-8f4837562e51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:41:01.869179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:01.875333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:41:01.876460 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:41:01.876460 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:01.876460 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:01.876460 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 626/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:01.876739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff9725-ef1c-4e0c-859a-c0fe223e94d1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:01.976738 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff9725-ef1c-4e0c-859a-c0fe223e94d1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:01.979749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:02.010609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3108d-864d-41c9-a5c5-f40aa98c83b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:02.011238 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3108d-864d-41c9-a5c5-f40aa98c83b2 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:02.011523 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3108d-864d-41c9-a5c5-f40aa98c83b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:02.011826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3108d-864d-41c9-a5c5-f40aa98c83b2 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:02.012469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3108d-864d-41c9-a5c5-f40aa98c83b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:02.012995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3108d-864d-41c9-a5c5-f40aa98c83b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:02.014963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3108d-864d-41c9-a5c5-f40aa98c83b2 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:02.026494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 603/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:02.032184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199993d5-dcd7-43c9-9872-8f4837562e51 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:02.041607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:02.130322 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199993d5-dcd7-43c9-9872-8f4837562e51 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:02.163691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e897c9-2be2-4af7-b7f2-97f57c544c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:02.164313 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e897c9-2be2-4af7-b7f2-97f57c544c22 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:02.164589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e897c9-2be2-4af7-b7f2-97f57c544c22 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:02.164859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e897c9-2be2-4af7-b7f2-97f57c544c22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:02.171988 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e897c9-2be2-4af7-b7f2-97f57c544c22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:02.173407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e897c9-2be2-4af7-b7f2-97f57c544c22 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-85e897c9-2be2-4af7-b7f2-97f57c544c22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:41:02.185085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:02.186112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:02.189623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:41:02.189623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:41:02.189623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:41:02.189623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:41:02.189623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:41:02.189623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:41:02.189623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:41:02.189623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:02.189623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 627/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:41:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:02.192427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3108d-864d-41c9-a5c5-f40aa98c83b2 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:02.292632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3108d-864d-41c9-a5c5-f40aa98c83b2 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:02.325646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41bac0-d822-4b61-8f44-740b93bf202a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:02.326248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41bac0-d822-4b61-8f44-740b93bf202a None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:02.326554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41bac0-d822-4b61-8f44-740b93bf202a None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:02.326832 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41bac0-d822-4b61-8f44-740b93bf202a None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:02.327510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41bac0-d822-4b61-8f44-740b93bf202a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:02.327874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41bac0-d822-4b61-8f44-740b93bf202a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:02.329293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41bac0-d822-4b61-8f44-740b93bf202a None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:02.341339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 604/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:41:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:02.347329 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e897c9-2be2-4af7-b7f2-97f57c544c22 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:02.410185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:02.444912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e897c9-2be2-4af7-b7f2-97f57c544c22 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:02.477622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1af1f-0757-464a-a455-91e60d480eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:02.479181 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1af1f-0757-464a-a455-91e60d480eea None admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:02.480189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1af1f-0757-464a-a455-91e60d480eea None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:02.480466 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1af1f-0757-464a-a455-91e60d480eea None admin] PATH_INFO: `/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/33e177d86450489cacc86578ba26509d/roles/4a5e4b92c86041c6aa22c4f4735b4a05` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:02.495037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1af1f-0757-464a-a455-91e60d480eea None admin] RBAC: Authorizing `identity:create_grant(project_id=6e2dc716277d42ae8316ebb2d36293a8, user_id=33e177d86450489cacc86578ba26509d, role_id=4a5e4b92c86041c6aa22c4f4735b4a05)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:02.496501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1af1f-0757-464a-a455-91e60d480eea None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:02.592404 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 628/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:41:02 2022] PUT /identity/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/33e177d86450489cacc86578ba26509d/roles/4a5e4b92c86041c6aa22c4f4735b4a05 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:41:04.736420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be81584-44be-4bb6-99ac-c031f93fadb7 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:04.736976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be81584-44be-4bb6-99ac-c031f93fadb7 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:04.736976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be81584-44be-4bb6-99ac-c031f93fadb7 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:04.738308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 605/1233] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:41:04.743933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8237b88-4b51-4061-8fc5-7a5a5d55ced3 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:04.744348 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8237b88-4b51-4061-8fc5-7a5a5d55ced3 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:04.744555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8237b88-4b51-4061-8fc5-7a5a5d55ced3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:04.763001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-a8237b88-4b51-4061-8fc5-7a5a5d55ced3 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:41:04.764167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a8237b88-4b51-4061-8fc5-7a5a5d55ced3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:41:04.831334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 629/1234] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:41:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:41:05.101148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be81584-44be-4bb6-99ac-c031f93fadb7 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:05.126883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:05.203019 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be81584-44be-4bb6-99ac-c031f93fadb7 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:05.236101 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a534fa0-e97d-4e81-8d41-abc7b9586109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:05.236721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a534fa0-e97d-4e81-8d41-abc7b9586109 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:05.237002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a534fa0-e97d-4e81-8d41-abc7b9586109 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:05.237276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a534fa0-e97d-4e81-8d41-abc7b9586109 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:05.247703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a534fa0-e97d-4e81-8d41-abc7b9586109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-5a534fa0-e97d-4e81-8d41-abc7b9586109 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:41:05.252411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:05.253522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:05.254408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:05.254408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:05.254408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:05.254408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:05.254408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:41:05.254408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:41:05.254408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:41:05.254408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:41:05.254408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:41:05.254408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:41:05.254408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:41:05.254408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:41:05.254408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 606/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Fri Sep 9 12:41:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:05.259941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8237b88-4b51-4061-8fc5-7a5a5d55ced3 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:05.305721 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:05.356889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8237b88-4b51-4061-8fc5-7a5a5d55ced3 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:05.390870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb6e42-993c-41fa-9a41-8a574e041f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:05.391473 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb6e42-993c-41fa-9a41-8a574e041f85 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:05.391754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb6e42-993c-41fa-9a41-8a574e041f85 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:05.392031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb6e42-993c-41fa-9a41-8a574e041f85 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:05.392726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb6e42-993c-41fa-9a41-8a574e041f85 None admin] RBAC: Adding query filter params (name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:05.393028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb6e42-993c-41fa-9a41-8a574e041f85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:05.394385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb6e42-993c-41fa-9a41-8a574e041f85 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:05.403268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 630/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Fri Sep 9 12:41:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:05.409222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a534fa0-e97d-4e81-8d41-abc7b9586109 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:05.492108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:05.506981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a534fa0-e97d-4e81-8d41-abc7b9586109 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:05.540288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1735ab0-7587-411e-b711-949fe9223871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:05.541951 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1735ab0-7587-411e-b711-949fe9223871 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:05.542231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1735ab0-7587-411e-b711-949fe9223871 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:05.542533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1735ab0-7587-411e-b711-949fe9223871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:05.552503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1735ab0-7587-411e-b711-949fe9223871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:05.553928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1735ab0-7587-411e-b711-949fe9223871 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-c1735ab0-7587-411e-b711-949fe9223871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:41:05.568653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:05.574461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:41:05.575641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:41:05.575641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:05.575641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:41:05.575641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 607/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:05.576004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb6e42-993c-41fa-9a41-8a574e041f85 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:05.674183 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb6e42-993c-41fa-9a41-8a574e041f85 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:05.682747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:05.706632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b94c7b-bed3-40d5-8d78-573e04ebd238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:05.707232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b94c7b-bed3-40d5-8d78-573e04ebd238 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:05.707511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b94c7b-bed3-40d5-8d78-573e04ebd238 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:05.707784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b94c7b-bed3-40d5-8d78-573e04ebd238 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:05.708425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b94c7b-bed3-40d5-8d78-573e04ebd238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:05.708726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b94c7b-bed3-40d5-8d78-573e04ebd238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:05.710073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b94c7b-bed3-40d5-8d78-573e04ebd238 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:05.721754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 631/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:41:05.727724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1735ab0-7587-411e-b711-949fe9223871 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:05.830339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1735ab0-7587-411e-b711-949fe9223871 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:05.864231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1772f9-d4bf-46db-a45d-0c06b4ac304b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:05.864854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1772f9-d4bf-46db-a45d-0c06b4ac304b None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:05.865155 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1772f9-d4bf-46db-a45d-0c06b4ac304b None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:05.865409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1772f9-d4bf-46db-a45d-0c06b4ac304b None admin] PATH_INFO: `/v3/users/placement` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:05.868058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:05.875617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1772f9-d4bf-46db-a45d-0c06b4ac304b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:05.877047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1772f9-d4bf-46db-a45d-0c06b4ac304b None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-4b1772f9-d4bf-46db-a45d-0c06b4ac304b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:41:05.888661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:05.889601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:05.890679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:05.890679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:05.890679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:41:05.890679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:41:05.890679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:41:05.890679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:41:05.890679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:41:05.890679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:41:05.890679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:41:05.890679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:41:05.890679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 608/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:05.895653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b94c7b-bed3-40d5-8d78-573e04ebd238 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:05.995234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b94c7b-bed3-40d5-8d78-573e04ebd238 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:06.028697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bfbfb-da5f-468c-988c-302ed888534c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:06.029285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bfbfb-da5f-468c-988c-302ed888534c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:06.029562 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bfbfb-da5f-468c-988c-302ed888534c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:06.029830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bfbfb-da5f-468c-988c-302ed888534c None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:06.030567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bfbfb-da5f-468c-988c-302ed888534c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:06.030867 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bfbfb-da5f-468c-988c-302ed888534c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:06.032260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bfbfb-da5f-468c-988c-302ed888534c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:06.051239 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 632/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Fri Sep 9 12:41:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:06.058171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1772f9-d4bf-46db-a45d-0c06b4ac304b None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:06.164682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1772f9-d4bf-46db-a45d-0c06b4ac304b None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:06.198124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e800a-1ebd-4ab9-9c03-345df76eadc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:06.198759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e800a-1ebd-4ab9-9c03-345df76eadc1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:06.199044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e800a-1ebd-4ab9-9c03-345df76eadc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:06.199314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e800a-1ebd-4ab9-9c03-345df76eadc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:06.260157 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:06.266374 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e800a-1ebd-4ab9-9c03-345df76eadc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:06.267833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e800a-1ebd-4ab9-9c03-345df76eadc1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:06.365714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 609/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:41:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 12:41:06.371581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bfbfb-da5f-468c-988c-302ed888534c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:06.397601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:06.470203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bfbfb-da5f-468c-988c-302ed888534c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:06.503393 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4becea5-a8e3-470b-8037-6391ab3abb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:06.504008 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4becea5-a8e3-470b-8037-6391ab3abb6e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:06.504282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4becea5-a8e3-470b-8037-6391ab3abb6e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:06.504552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4becea5-a8e3-470b-8037-6391ab3abb6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:06.517509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4becea5-a8e3-470b-8037-6391ab3abb6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:06.517509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4becea5-a8e3-470b-8037-6391ab3abb6e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-c4becea5-a8e3-470b-8037-6391ab3abb6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:41:06.530849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:06.531853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:41:06.532744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:41:06.532744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:06.532744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:06.532744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 633/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:06.537738 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e800a-1ebd-4ab9-9c03-345df76eadc1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:06.628412 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:06.634442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e800a-1ebd-4ab9-9c03-345df76eadc1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:06.666971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86683373-4f51-403c-bdf2-4eeb6a4968c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:06.667593 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86683373-4f51-403c-bdf2-4eeb6a4968c1 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:06.667870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86683373-4f51-403c-bdf2-4eeb6a4968c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:06.668239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86683373-4f51-403c-bdf2-4eeb6a4968c1 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:06.668831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86683373-4f51-403c-bdf2-4eeb6a4968c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:06.669175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86683373-4f51-403c-bdf2-4eeb6a4968c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:06.670743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86683373-4f51-403c-bdf2-4eeb6a4968c1 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:06.681757 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 610/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:06.687797 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4becea5-a8e3-470b-8037-6391ab3abb6e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:06.764926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:06.785668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4becea5-a8e3-470b-8037-6391ab3abb6e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:06.816877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e4339-9485-4a88-9cd7-5f4b13689ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:06.817497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e4339-9485-4a88-9cd7-5f4b13689ac4 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:06.817765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e4339-9485-4a88-9cd7-5f4b13689ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:06.818029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e4339-9485-4a88-9cd7-5f4b13689ac4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:06.824923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e4339-9485-4a88-9cd7-5f4b13689ac4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:06.826324 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e4339-9485-4a88-9cd7-5f4b13689ac4 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-089e4339-9485-4a88-9cd7-5f4b13689ac4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:41:06.837992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:06.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:06.839827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:41:06.839827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:41:06.839827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:41:06.839827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:41:06.839827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:41:06.839827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:41:06.839827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:41:06.839827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:06.839827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 634/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:41:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:41:06.845087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86683373-4f51-403c-bdf2-4eeb6a4968c1 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:06.941205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86683373-4f51-403c-bdf2-4eeb6a4968c1 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:06.973511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4298b-5cb0-4e4d-9cf5-ae759adf32af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:06.974102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4298b-5cb0-4e4d-9cf5-ae759adf32af None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:06.974378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4298b-5cb0-4e4d-9cf5-ae759adf32af None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:06.974675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4298b-5cb0-4e4d-9cf5-ae759adf32af None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:06.975344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4298b-5cb0-4e4d-9cf5-ae759adf32af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:06.975644 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4298b-5cb0-4e4d-9cf5-ae759adf32af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:06.977046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4298b-5cb0-4e4d-9cf5-ae759adf32af None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:06.988430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 611/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:41:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:06.995469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e4339-9485-4a88-9cd7-5f4b13689ac4 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:07.093627 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e4339-9485-4a88-9cd7-5f4b13689ac4 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:07.126731 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094e2b4-99b2-4d06-98c8-ce92b409e995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:07.127333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094e2b4-99b2-4d06-98c8-ce92b409e995 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:07.127608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094e2b4-99b2-4d06-98c8-ce92b409e995 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:07.127933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094e2b4-99b2-4d06-98c8-ce92b409e995 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:07.128657 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c094e2b4-99b2-4d06-98c8-ce92b409e995 None admin] RBAC: Adding query filter params (role.id=4a5e4b92c86041c6aa22c4f4735b4a05, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=33e177d86450489cacc86578ba26509d) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:07.128965 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c094e2b4-99b2-4d06-98c8-ce92b409e995 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:07.130372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c094e2b4-99b2-4d06-98c8-ce92b409e995 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:07.136640 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 635/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:41:06 2022] GET /identity/v3/role_assignments?user.id=33e177d86450489cacc86578ba26509d&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=4a5e4b92c86041c6aa22c4f4735b4a05 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:09.385198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7c022-ac30-4194-9343-8d15e63e9507 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:09.385712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7c022-ac30-4194-9343-8d15e63e9507 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:09.385712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7c022-ac30-4194-9343-8d15e63e9507 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:09.387046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 612/1247] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:41:09.392333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bb0e8-1655-4f89-9db3-8297feea1455 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:09.392777 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bb0e8-1655-4f89-9db3-8297feea1455 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:09.392885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bb0e8-1655-4f89-9db3-8297feea1455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:09.399305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:09.412075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-355bb0e8-1655-4f89-9db3-8297feea1455 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:41:09.413244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-355bb0e8-1655-4f89-9db3-8297feea1455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:41:09.477042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 636/1248] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:41:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:41:09.745369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7c022-ac30-4194-9343-8d15e63e9507 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:09.763273 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:09.851658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7c022-ac30-4194-9343-8d15e63e9507 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:09.885620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3512738-bd91-4cf1-a7ea-0a230ca91984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:09.886245 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512738-bd91-4cf1-a7ea-0a230ca91984 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:09.886561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512738-bd91-4cf1-a7ea-0a230ca91984 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:09.886837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512738-bd91-4cf1-a7ea-0a230ca91984 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:09.896709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3512738-bd91-4cf1-a7ea-0a230ca91984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-e3512738-bd91-4cf1-a7ea-0a230ca91984 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:41:09.902205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:09.903346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:09.904367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:09.904367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:09.904367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:09.904367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:09.904367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:41:09.904367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:41:09.904367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:41:09.904367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:41:09.904367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:41:09.904367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:41:09.904367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:41:09.904367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:41:09.906635 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 613/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:41:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:09.909391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bb0e8-1655-4f89-9db3-8297feea1455 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:10.005281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bb0e8-1655-4f89-9db3-8297feea1455 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:10.036744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:10.042401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1aed8-08ac-4fad-8aef-151bf7adaa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:10.043032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1aed8-08ac-4fad-8aef-151bf7adaa00 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:10.043306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1aed8-08ac-4fad-8aef-151bf7adaa00 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:10.043571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1aed8-08ac-4fad-8aef-151bf7adaa00 None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:10.044223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1aed8-08ac-4fad-8aef-151bf7adaa00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:10.044519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1aed8-08ac-4fad-8aef-151bf7adaa00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:10.045851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1aed8-08ac-4fad-8aef-151bf7adaa00 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:10.053574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 637/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:41:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:10.059650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3512738-bd91-4cf1-a7ea-0a230ca91984 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:10.138053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:10.161784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3512738-bd91-4cf1-a7ea-0a230ca91984 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:10.196508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d2230-d92a-4ac6-8590-76e0639840df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:10.197162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d2230-d92a-4ac6-8590-76e0639840df None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:10.197443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d2230-d92a-4ac6-8590-76e0639840df None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:10.197789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d2230-d92a-4ac6-8590-76e0639840df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:10.207454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d2230-d92a-4ac6-8590-76e0639840df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:10.208898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d2230-d92a-4ac6-8590-76e0639840df None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-bd3d2230-d92a-4ac6-8590-76e0639840df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:41:10.223775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:10.224845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:41:10.225760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:41:10.225760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:10.225760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:41:10.225760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 614/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:10.232084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1aed8-08ac-4fad-8aef-151bf7adaa00 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:10.333669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1aed8-08ac-4fad-8aef-151bf7adaa00 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:10.367639 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87af3fc-580f-4f10-beea-c91f86a9bbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:10.369891 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87af3fc-580f-4f10-beea-c91f86a9bbd8 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:10.369891 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87af3fc-580f-4f10-beea-c91f86a9bbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:10.369891 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87af3fc-580f-4f10-beea-c91f86a9bbd8 None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:10.370189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87af3fc-580f-4f10-beea-c91f86a9bbd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:10.370378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87af3fc-580f-4f10-beea-c91f86a9bbd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:10.371761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87af3fc-580f-4f10-beea-c91f86a9bbd8 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:10.383254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 638/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:10.389423 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d2230-d92a-4ac6-8590-76e0639840df None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:10.487248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d2230-d92a-4ac6-8590-76e0639840df None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:10.513081 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:10.520597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4bf6a-2c99-4010-b213-803a778eb65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:10.521222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4bf6a-2c99-4010-b213-803a778eb65d None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:10.521501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4bf6a-2c99-4010-b213-803a778eb65d None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:10.521772 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4bf6a-2c99-4010-b213-803a778eb65d None admin] PATH_INFO: `/v3/users/placement` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:10.528988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4bf6a-2c99-4010-b213-803a778eb65d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:10.530405 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4bf6a-2c99-4010-b213-803a778eb65d None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-abb4bf6a-2c99-4010-b213-803a778eb65d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:41:10.541765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:10.550938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:10.552172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:10.552172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:10.552172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:41:10.552172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:41:10.552172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:41:10.552172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:41:10.552172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:41:10.552172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:41:10.552172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:41:10.552172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:41:10.552172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 615/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:10.552890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87af3fc-580f-4f10-beea-c91f86a9bbd8 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:10.602450 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:10.644473 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87af3fc-580f-4f10-beea-c91f86a9bbd8 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:10.676723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63407c67-9dee-4cf0-b30c-291b44768e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:10.677318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63407c67-9dee-4cf0-b30c-291b44768e74 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:10.677592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63407c67-9dee-4cf0-b30c-291b44768e74 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:10.677858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63407c67-9dee-4cf0-b30c-291b44768e74 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:10.678610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63407c67-9dee-4cf0-b30c-291b44768e74 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:10.678911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63407c67-9dee-4cf0-b30c-291b44768e74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:10.680315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63407c67-9dee-4cf0-b30c-291b44768e74 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:10.696522 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 639/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Fri Sep 9 12:41:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:10.702521 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4bf6a-2c99-4010-b213-803a778eb65d None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:10.798586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4bf6a-2c99-4010-b213-803a778eb65d None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:10.830570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c961a2-be73-4782-ae1e-887d0ba9cbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:10.831167 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c961a2-be73-4782-ae1e-887d0ba9cbad None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:10.831443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c961a2-be73-4782-ae1e-887d0ba9cbad None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:10.831712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c961a2-be73-4782-ae1e-887d0ba9cbad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:10.891203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:10.903274 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c961a2-be73-4782-ae1e-887d0ba9cbad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:10.904711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c961a2-be73-4782-ae1e-887d0ba9cbad None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:11.009146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 616/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:41:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:41:11.017700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63407c67-9dee-4cf0-b30c-291b44768e74 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:11.135564 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63407c67-9dee-4cf0-b30c-291b44768e74 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:11.149784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:11.179916 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f529d08-4f77-4818-bc7e-239086cd6b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:11.181547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f529d08-4f77-4818-bc7e-239086cd6b9d None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:11.181547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f529d08-4f77-4818-bc7e-239086cd6b9d None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:11.181547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f529d08-4f77-4818-bc7e-239086cd6b9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:11.193800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f529d08-4f77-4818-bc7e-239086cd6b9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:11.196576 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f529d08-4f77-4818-bc7e-239086cd6b9d None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-9f529d08-4f77-4818-bc7e-239086cd6b9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:41:11.213634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:11.215168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:41:11.216443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:41:11.216443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:11.216443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:11.216443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 640/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:11.224271 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c961a2-be73-4782-ae1e-887d0ba9cbad None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:11.325380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:11.342707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c961a2-be73-4782-ae1e-887d0ba9cbad None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:11.393200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83115f-4785-4f8c-96bd-7fd042646f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:11.394112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83115f-4785-4f8c-96bd-7fd042646f2e None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:11.394617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83115f-4785-4f8c-96bd-7fd042646f2e None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:11.395061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83115f-4785-4f8c-96bd-7fd042646f2e None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:11.396138 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83115f-4785-4f8c-96bd-7fd042646f2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:11.396647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83115f-4785-4f8c-96bd-7fd042646f2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:11.399016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83115f-4785-4f8c-96bd-7fd042646f2e None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:11.417128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 617/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:11.421182 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f529d08-4f77-4818-bc7e-239086cd6b9d None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:11.546626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f529d08-4f77-4818-bc7e-239086cd6b9d None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:11.586607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d8857-66eb-4dd8-a0d9-39529f09c6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:11.587281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d8857-66eb-4dd8-a0d9-39529f09c6da None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:11.587494 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d8857-66eb-4dd8-a0d9-39529f09c6da None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:11.588285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d8857-66eb-4dd8-a0d9-39529f09c6da None admin] PATH_INFO: `/v3/projects/service` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:11.597185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d8857-66eb-4dd8-a0d9-39529f09c6da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:11.599665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d8857-66eb-4dd8-a0d9-39529f09c6da None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-692d8857-66eb-4dd8-a0d9-39529f09c6da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:41:11.614955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:11.616076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:11.617175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:41:11.617175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:41:11.617175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:41:11.617175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:41:11.617175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:41:11.617175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:41:11.617175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:41:11.617175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:11.617175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 641/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:41:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:11.622099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83115f-4785-4f8c-96bd-7fd042646f2e None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:11.741427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83115f-4785-4f8c-96bd-7fd042646f2e None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:11.779855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd660b-67de-43ea-bc5f-205af8839ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:11.780464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd660b-67de-43ea-bc5f-205af8839ea3 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:11.780745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd660b-67de-43ea-bc5f-205af8839ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:11.781018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd660b-67de-43ea-bc5f-205af8839ea3 None admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:11.781705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd660b-67de-43ea-bc5f-205af8839ea3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:11.782010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd660b-67de-43ea-bc5f-205af8839ea3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:11.783506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd660b-67de-43ea-bc5f-205af8839ea3 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:11.799192 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 618/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:41:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:11.806996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d8857-66eb-4dd8-a0d9-39529f09c6da None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:11.845984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:11.916680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d8857-66eb-4dd8-a0d9-39529f09c6da None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:11.949039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5aa81-f4d2-45b0-b74e-e35f134a8e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:11.949632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5aa81-f4d2-45b0-b74e-e35f134a8e2e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:11.949905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5aa81-f4d2-45b0-b74e-e35f134a8e2e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:11.950172 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5aa81-f4d2-45b0-b74e-e35f134a8e2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:11.950919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5aa81-f4d2-45b0-b74e-e35f134a8e2e None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=33e177d86450489cacc86578ba26509d) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:11.951219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5aa81-f4d2-45b0-b74e-e35f134a8e2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:11.952679 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5aa81-f4d2-45b0-b74e-e35f134a8e2e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:11.958258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 642/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:41:11 2022] GET /identity/v3/role_assignments?user.id=33e177d86450489cacc86578ba26509d&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:14.106962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5215dba-5a2e-4051-8388-4293dbede5f9 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:14.107489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5215dba-5a2e-4051-8388-4293dbede5f9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:14.107489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5215dba-5a2e-4051-8388-4293dbede5f9 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:14.108835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 619/1261] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:41:14.113911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad5742-70ce-4056-b79a-fc24ab1ebd53 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:14.114304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad5742-70ce-4056-b79a-fc24ab1ebd53 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:14.114453 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad5742-70ce-4056-b79a-fc24ab1ebd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:14.133611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-07ad5742-70ce-4056-b79a-fc24ab1ebd53 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:41:14.134881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-07ad5742-70ce-4056-b79a-fc24ab1ebd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:41:14.205474 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 643/1262] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:41:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:41:14.480428 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5215dba-5a2e-4051-8388-4293dbede5f9 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:14.491163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:14.596250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5215dba-5a2e-4051-8388-4293dbede5f9 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:14.630421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b4cd9-2118-46d3-a063-6e3ed780b574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:14.631067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b4cd9-2118-46d3-a063-6e3ed780b574 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:14.631343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b4cd9-2118-46d3-a063-6e3ed780b574 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:14.631611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b4cd9-2118-46d3-a063-6e3ed780b574 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:14.641268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b4cd9-2118-46d3-a063-6e3ed780b574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-c15b4cd9-2118-46d3-a063-6e3ed780b574 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:41:14.645945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:14.647171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:14.648444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:14.648444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:14.648444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:14.648444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:14.648444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:41:14.648444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:41:14.648444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:41:14.648444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:41:14.648444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:41:14.648444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:41:14.648444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:41:14.648444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:41:14.648444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 620/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:41:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:14.653684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad5742-70ce-4056-b79a-fc24ab1ebd53 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:14.672658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:14.748959 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad5742-70ce-4056-b79a-fc24ab1ebd53 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:14.783104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab41bb9-a9df-4d62-b1da-3ac1fd4185ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:14.783730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab41bb9-a9df-4d62-b1da-3ac1fd4185ad None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:14.784009 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab41bb9-a9df-4d62-b1da-3ac1fd4185ad None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:14.784283 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab41bb9-a9df-4d62-b1da-3ac1fd4185ad None admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:14.785046 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab41bb9-a9df-4d62-b1da-3ac1fd4185ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:14.785350 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab41bb9-a9df-4d62-b1da-3ac1fd4185ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:14.786730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab41bb9-a9df-4d62-b1da-3ac1fd4185ad None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:14.794953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 644/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:41:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:14.801019 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b4cd9-2118-46d3-a063-6e3ed780b574 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:14.869202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:14.907619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b4cd9-2118-46d3-a063-6e3ed780b574 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:14.940232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b20af-5714-42b8-b00f-3a414a939593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:14.941048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b20af-5714-42b8-b00f-3a414a939593 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:14.941538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b20af-5714-42b8-b00f-3a414a939593 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:14.941961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b20af-5714-42b8-b00f-3a414a939593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:14.953057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b20af-5714-42b8-b00f-3a414a939593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:14.954504 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b20af-5714-42b8-b00f-3a414a939593 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-eb9b20af-5714-42b8-b00f-3a414a939593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:41:14.969249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:14.970229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:41:14.981256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:41:14.981256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:14.981256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:41:14.981256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 621/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:41:14.981536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab41bb9-a9df-4d62-b1da-3ac1fd4185ad None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:15.056724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:15.076204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab41bb9-a9df-4d62-b1da-3ac1fd4185ad None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:15.109343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa17abd-15dc-42f8-a3a0-c580de70214c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:15.109945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa17abd-15dc-42f8-a3a0-c580de70214c None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:15.110245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa17abd-15dc-42f8-a3a0-c580de70214c None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:15.110519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa17abd-15dc-42f8-a3a0-c580de70214c None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:15.111164 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa17abd-15dc-42f8-a3a0-c580de70214c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:15.111466 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa17abd-15dc-42f8-a3a0-c580de70214c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:15.112823 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa17abd-15dc-42f8-a3a0-c580de70214c None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:15.125133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 645/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:15.130970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b20af-5714-42b8-b00f-3a414a939593 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:15.229625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b20af-5714-42b8-b00f-3a414a939593 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:15.248791 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:15.262677 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852f1c4-3b97-4800-be5d-8826dbaca131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:15.263349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852f1c4-3b97-4800-be5d-8826dbaca131 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:15.263629 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852f1c4-3b97-4800-be5d-8826dbaca131 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:15.263902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852f1c4-3b97-4800-be5d-8826dbaca131 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:15.272000 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852f1c4-3b97-4800-be5d-8826dbaca131 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:15.273420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852f1c4-3b97-4800-be5d-8826dbaca131 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-4852f1c4-3b97-4800-be5d-8826dbaca131 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:41:15.284866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:15.286027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:15.287260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:15.287260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:15.287260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:41:15.287260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:41:15.287260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:41:15.287260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:41:15.287260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:41:15.287260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:41:15.287260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:41:15.287260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:41:15.287260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 622/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:15.292759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa17abd-15dc-42f8-a3a0-c580de70214c None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:15.392077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa17abd-15dc-42f8-a3a0-c580de70214c None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:15.426265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa29768-5dfa-4e6c-bcee-bbbcce13e424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:15.426940 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa29768-5dfa-4e6c-bcee-bbbcce13e424 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:15.427225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa29768-5dfa-4e6c-bcee-bbbcce13e424 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:15.427501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa29768-5dfa-4e6c-bcee-bbbcce13e424 None admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:15.428298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa29768-5dfa-4e6c-bcee-bbbcce13e424 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:15.428603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa29768-5dfa-4e6c-bcee-bbbcce13e424 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:15.430063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa29768-5dfa-4e6c-bcee-bbbcce13e424 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:15.439256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:15.447756 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 646/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Fri Sep 9 12:41:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:15.454954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852f1c4-3b97-4800-be5d-8826dbaca131 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:15.553684 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852f1c4-3b97-4800-be5d-8826dbaca131 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:15.587530 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081fac6-009b-4453-b461-8663ef95d820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:15.588812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081fac6-009b-4453-b461-8663ef95d820 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:15.589090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081fac6-009b-4453-b461-8663ef95d820 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:15.589363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081fac6-009b-4453-b461-8663ef95d820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:15.599838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081fac6-009b-4453-b461-8663ef95d820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:15.601281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081fac6-009b-4453-b461-8663ef95d820 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-5081fac6-009b-4453-b461-8663ef95d820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:41:15.615367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:15.616503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:41:15.617443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:41:15.617443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:15.617443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:41:15.617443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 623/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:15.622459 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa29768-5dfa-4e6c-bcee-bbbcce13e424 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:15.720961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa29768-5dfa-4e6c-bcee-bbbcce13e424 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:15.754828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4361f9-1ae8-409a-b26c-7684a39be17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:15.755380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4361f9-1ae8-409a-b26c-7684a39be17f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:15.755660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4361f9-1ae8-409a-b26c-7684a39be17f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:15.755933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4361f9-1ae8-409a-b26c-7684a39be17f None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:15.756685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4361f9-1ae8-409a-b26c-7684a39be17f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:15.757008 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4361f9-1ae8-409a-b26c-7684a39be17f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:15.758331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4361f9-1ae8-409a-b26c-7684a39be17f None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:15.770130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 647/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:15.776622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081fac6-009b-4453-b461-8663ef95d820 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:15.806806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:15.871742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081fac6-009b-4453-b461-8663ef95d820 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:15.905202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb1a7f-03b6-46cc-a8e2-f420c96bd461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:15.905819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb1a7f-03b6-46cc-a8e2-f420c96bd461 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:15.906096 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb1a7f-03b6-46cc-a8e2-f420c96bd461 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:15.906448 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb1a7f-03b6-46cc-a8e2-f420c96bd461 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:15.914968 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb1a7f-03b6-46cc-a8e2-f420c96bd461 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:15.916535 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb1a7f-03b6-46cc-a8e2-f420c96bd461 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-59cb1a7f-03b6-46cc-a8e2-f420c96bd461 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:41:15.927469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:15.928620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:15.934389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:41:15.934389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:41:15.934389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:41:15.934389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:41:15.934389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:41:15.934389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:41:15.934389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:41:15.934389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:41:15.934389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 624/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:41:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:15.935379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4361f9-1ae8-409a-b26c-7684a39be17f None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:15.992834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:16.033331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4361f9-1ae8-409a-b26c-7684a39be17f None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:16.066232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bcfaa-1b57-4b0a-a779-eca9a793dd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:16.066867 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bcfaa-1b57-4b0a-a779-eca9a793dd5f None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:16.067149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bcfaa-1b57-4b0a-a779-eca9a793dd5f None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:16.067420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bcfaa-1b57-4b0a-a779-eca9a793dd5f None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:16.068111 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bcfaa-1b57-4b0a-a779-eca9a793dd5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:16.068416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bcfaa-1b57-4b0a-a779-eca9a793dd5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:16.069831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bcfaa-1b57-4b0a-a779-eca9a793dd5f None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:16.083854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 648/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:41:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:41:16.089932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb1a7f-03b6-46cc-a8e2-f420c96bd461 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:16.178494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:16.187893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb1a7f-03b6-46cc-a8e2-f420c96bd461 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:16.221590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdfe61-124b-4834-9f2c-467d0d151dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:16.222245 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdfe61-124b-4834-9f2c-467d0d151dae None admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:16.222556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdfe61-124b-4834-9f2c-467d0d151dae None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:16.222833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdfe61-124b-4834-9f2c-467d0d151dae None admin] PATH_INFO: `/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/33e177d86450489cacc86578ba26509d/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:16.224706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdfe61-124b-4834-9f2c-467d0d151dae None admin] RBAC: Authorizing `identity:create_grant(project_id=6e2dc716277d42ae8316ebb2d36293a8, user_id=33e177d86450489cacc86578ba26509d, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:16.226179 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdfe61-124b-4834-9f2c-467d0d151dae None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:16.323330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 625/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Fri Sep 9 12:41:16 2022] PUT /identity/v3/projects/6e2dc716277d42ae8316ebb2d36293a8/users/33e177d86450489cacc86578ba26509d/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:41:18.419806 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd159154-feb3-49d2-91a3-ee50611d771c None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:18.420351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd159154-feb3-49d2-91a3-ee50611d771c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:18.420351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd159154-feb3-49d2-91a3-ee50611d771c None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:18.421588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 649/1274] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:41:18.426689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15513bd3-644f-47b1-89e3-e01d4fbc44e1 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:18.427090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15513bd3-644f-47b1-89e3-e01d4fbc44e1 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:18.427247 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15513bd3-644f-47b1-89e3-e01d4fbc44e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:18.446314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-15513bd3-644f-47b1-89e3-e01d4fbc44e1 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:41:18.448269 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-15513bd3-644f-47b1-89e3-e01d4fbc44e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:41:18.512806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 626/1275] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:41:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:41:18.781932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd159154-feb3-49d2-91a3-ee50611d771c None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:18.894096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd159154-feb3-49d2-91a3-ee50611d771c None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:18.911833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:18.928440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32860345-891c-4a37-81c9-d21a613b854b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:18.929061 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32860345-891c-4a37-81c9-d21a613b854b None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:18.929337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32860345-891c-4a37-81c9-d21a613b854b None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:18.929614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32860345-891c-4a37-81c9-d21a613b854b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:18.939858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32860345-891c-4a37-81c9-d21a613b854b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-32860345-891c-4a37-81c9-d21a613b854b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:41:18.944608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:18.948223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:18.949305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:18.949305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:18.949305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:18.949305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:18.949305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:41:18.949305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:41:18.949305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:41:18.949305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:41:18.949305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:41:18.949305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:41:18.949305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:41:18.949305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:18.949305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 650/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Fri Sep 9 12:41:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:18.951784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15513bd3-644f-47b1-89e3-e01d4fbc44e1 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:19.030918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:19.052265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15513bd3-644f-47b1-89e3-e01d4fbc44e1 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:19.084960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fabcc9-fa36-46a7-b22c-b1b9fd3b86fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:19.085557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fabcc9-fa36-46a7-b22c-b1b9fd3b86fd None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:19.085846 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fabcc9-fa36-46a7-b22c-b1b9fd3b86fd None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:19.086121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fabcc9-fa36-46a7-b22c-b1b9fd3b86fd None admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:19.087023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fabcc9-fa36-46a7-b22c-b1b9fd3b86fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:19.088002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fabcc9-fa36-46a7-b22c-b1b9fd3b86fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:19.089774 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fabcc9-fa36-46a7-b22c-b1b9fd3b86fd None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:19.097685 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 627/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Fri Sep 9 12:41:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:19.104211 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32860345-891c-4a37-81c9-d21a613b854b None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:19.204041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32860345-891c-4a37-81c9-d21a613b854b None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:19.237512 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5adfd8-ec41-4d84-a3d1-daf6481bc847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:19.238124 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5adfd8-ec41-4d84-a3d1-daf6481bc847 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:19.238398 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5adfd8-ec41-4d84-a3d1-daf6481bc847 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:19.238695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5adfd8-ec41-4d84-a3d1-daf6481bc847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:19.248299 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5adfd8-ec41-4d84-a3d1-daf6481bc847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:19.250540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5adfd8-ec41-4d84-a3d1-daf6481bc847 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-ed5adfd8-ec41-4d84-a3d1-daf6481bc847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:41:19.264075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:19.265202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:41:19.266452 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:41:19.266452 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:19.266452 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:19.266452 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 651/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:41:19.271014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fabcc9-fa36-46a7-b22c-b1b9fd3b86fd None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:19.372729 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fabcc9-fa36-46a7-b22c-b1b9fd3b86fd None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:19.407744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808047a-c445-4972-8e3c-7d8a252cacc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:19.408343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808047a-c445-4972-8e3c-7d8a252cacc7 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:19.408621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808047a-c445-4972-8e3c-7d8a252cacc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:19.408894 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808047a-c445-4972-8e3c-7d8a252cacc7 None admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:19.409522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808047a-c445-4972-8e3c-7d8a252cacc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:19.409828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808047a-c445-4972-8e3c-7d8a252cacc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:19.411203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808047a-c445-4972-8e3c-7d8a252cacc7 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:19.423182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 628/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:19.428844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5adfd8-ec41-4d84-a3d1-daf6481bc847 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:19.461679 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:19.537786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5adfd8-ec41-4d84-a3d1-daf6481bc847 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:19.575141 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6332d92-5a43-41f8-852b-2a7fc77be023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:19.575814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6332d92-5a43-41f8-852b-2a7fc77be023 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:19.576093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6332d92-5a43-41f8-852b-2a7fc77be023 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:19.576421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6332d92-5a43-41f8-852b-2a7fc77be023 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:19.584536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6332d92-5a43-41f8-852b-2a7fc77be023 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:19.585985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6332d92-5a43-41f8-852b-2a7fc77be023 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-c6332d92-5a43-41f8-852b-2a7fc77be023 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:41:19.598625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:19.599825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:19.600817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:19.600817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:19.600817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:41:19.600817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:41:19.600817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:41:19.600817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:41:19.600817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:41:19.600817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:41:19.600817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:41:19.600817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:19.600817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 652/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:19.606947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808047a-c445-4972-8e3c-7d8a252cacc7 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:19.610836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:19.715010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808047a-c445-4972-8e3c-7d8a252cacc7 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:19.751469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b601a8-0b7d-4a7d-8a88-a37e863b1344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:19.752109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b601a8-0b7d-4a7d-8a88-a37e863b1344 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:19.752389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b601a8-0b7d-4a7d-8a88-a37e863b1344 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:19.752659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b601a8-0b7d-4a7d-8a88-a37e863b1344 None admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:19.753390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b601a8-0b7d-4a7d-8a88-a37e863b1344 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:19.753695 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b601a8-0b7d-4a7d-8a88-a37e863b1344 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:19.755180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b601a8-0b7d-4a7d-8a88-a37e863b1344 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:19.774048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 629/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Fri Sep 9 12:41:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:19.780108 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6332d92-5a43-41f8-852b-2a7fc77be023 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:19.871476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:19.880706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6332d92-5a43-41f8-852b-2a7fc77be023 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:19.915619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f348a2-5a4c-4d8a-b416-5f16f447a0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:19.916260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f348a2-5a4c-4d8a-b416-5f16f447a0c5 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:19.916535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f348a2-5a4c-4d8a-b416-5f16f447a0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:19.916801 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f348a2-5a4c-4d8a-b416-5f16f447a0c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:19.982419 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f348a2-5a4c-4d8a-b416-5f16f447a0c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:19.983799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f348a2-5a4c-4d8a-b416-5f16f447a0c5 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:20.068067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:20.077733 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 653/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Fri Sep 9 12:41:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:41:20.083669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b601a8-0b7d-4a7d-8a88-a37e863b1344 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:20.146667 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:20.184894 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b601a8-0b7d-4a7d-8a88-a37e863b1344 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:20.218129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd180f8-30da-45eb-b17e-c3230395c302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:20.218790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd180f8-30da-45eb-b17e-c3230395c302 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:20.219075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd180f8-30da-45eb-b17e-c3230395c302 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:20.219351 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd180f8-30da-45eb-b17e-c3230395c302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:20.229670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd180f8-30da-45eb-b17e-c3230395c302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:20.231219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd180f8-30da-45eb-b17e-c3230395c302 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-ccd180f8-30da-45eb-b17e-c3230395c302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:41:20.245980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:20.247116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:41:20.248134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:41:20.248134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:41:20.248134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:41:20.248134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 630/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:20.252873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f348a2-5a4c-4d8a-b416-5f16f447a0c5 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:20.347142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f348a2-5a4c-4d8a-b416-5f16f447a0c5 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:20.380471 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3827c7-a75b-4589-b456-bfb53086c9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:20.381063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3827c7-a75b-4589-b456-bfb53086c9fb None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:20.381347 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3827c7-a75b-4589-b456-bfb53086c9fb None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:20.381618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3827c7-a75b-4589-b456-bfb53086c9fb None admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:20.382252 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3827c7-a75b-4589-b456-bfb53086c9fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:20.382646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3827c7-a75b-4589-b456-bfb53086c9fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:20.383995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3827c7-a75b-4589-b456-bfb53086c9fb None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:20.395073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 654/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Fri Sep 9 12:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:20.401428 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd180f8-30da-45eb-b17e-c3230395c302 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:20.498513 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd180f8-30da-45eb-b17e-c3230395c302 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:20.511881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:20.531551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f55f7-11b3-4b85-ad9a-0e795e2a2d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:20.532164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f55f7-11b3-4b85-ad9a-0e795e2a2d4c None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:20.532442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f55f7-11b3-4b85-ad9a-0e795e2a2d4c None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:20.532711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f55f7-11b3-4b85-ad9a-0e795e2a2d4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:20.538960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f55f7-11b3-4b85-ad9a-0e795e2a2d4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:20.540370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f55f7-11b3-4b85-ad9a-0e795e2a2d4c None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application [None req-1b3f55f7-11b3-4b85-ad9a-0e795e2a2d4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:41:20.552526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:20.555048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:20.556312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:41:20.556312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:41:20.556312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:41:20.556312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:41:20.556312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:41:20.556312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:41:20.556312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:41:20.556312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: ERROR keystone.server.flask.application Sep 09 12:41:20.556312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 631/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Fri Sep 9 12:41:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:20.559570 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3827c7-a75b-4589-b456-bfb53086c9fb None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:20.609096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:20.658347 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3827c7-a75b-4589-b456-bfb53086c9fb None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:20.691578 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441db94-546e-4de3-9c27-d2a5d1e139ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:20.692177 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441db94-546e-4de3-9c27-d2a5d1e139ac None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:20.692451 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441db94-546e-4de3-9c27-d2a5d1e139ac None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:20.692718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441db94-546e-4de3-9c27-d2a5d1e139ac None admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:20.693401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441db94-546e-4de3-9c27-d2a5d1e139ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:20.693696 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441db94-546e-4de3-9c27-d2a5d1e139ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:20.695125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441db94-546e-4de3-9c27-d2a5d1e139ac None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:20.707397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 655/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Fri Sep 9 12:41:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:20.714379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f55f7-11b3-4b85-ad9a-0e795e2a2d4c None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:20.812432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f55f7-11b3-4b85-ad9a-0e795e2a2d4c None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:20.845816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffbf44-1435-4a67-9a37-cb1ed4a2a100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:20.846407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffbf44-1435-4a67-9a37-cb1ed4a2a100 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:20.846715 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffbf44-1435-4a67-9a37-cb1ed4a2a100 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:20.846991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffbf44-1435-4a67-9a37-cb1ed4a2a100 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:20.847694 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffbf44-1435-4a67-9a37-cb1ed4a2a100 None admin] RBAC: Adding query filter params (role.id=a2103068c87a47748ba790b7ea0dd26c, scope.project.id=6e2dc716277d42ae8316ebb2d36293a8, user.id=33e177d86450489cacc86578ba26509d) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:20.847998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffbf44-1435-4a67-9a37-cb1ed4a2a100 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:20.849419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffbf44-1435-4a67-9a37-cb1ed4a2a100 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:20.855234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 632/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Fri Sep 9 12:41:20 2022] GET /identity/v3/role_assignments?user.id=33e177d86450489cacc86578ba26509d&scope.project.id=6e2dc716277d42ae8316ebb2d36293a8&role.id=a2103068c87a47748ba790b7ea0dd26c => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:23.043937 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaadf82-47e3-4ff8-a44e-c9e92e86c138 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:23.044493 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaadf82-47e3-4ff8-a44e-c9e92e86c138 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:23.044493 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaadf82-47e3-4ff8-a44e-c9e92e86c138 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:23.045819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 656/1288] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:41:23.051110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f43de3-1be0-4cdc-a089-a74609bc8d84 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:23.051590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f43de3-1be0-4cdc-a089-a74609bc8d84 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:23.051701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f43de3-1be0-4cdc-a089-a74609bc8d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:23.069984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-26f43de3-1be0-4cdc-a089-a74609bc8d84 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:41:23.071209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-26f43de3-1be0-4cdc-a089-a74609bc8d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:41:23.101016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:23.138274 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 633/1289] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:41:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:41:23.409438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaadf82-47e3-4ff8-a44e-c9e92e86c138 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:23.506069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaadf82-47e3-4ff8-a44e-c9e92e86c138 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:23.516314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:23.538978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32e6da-029f-420b-a7b7-64fcd87d1fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:23.539588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32e6da-029f-420b-a7b7-64fcd87d1fee None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:23.539860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32e6da-029f-420b-a7b7-64fcd87d1fee None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:23.540124 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32e6da-029f-420b-a7b7-64fcd87d1fee None admin] PATH_INFO: `/v3/services/placement` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:23.541348 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32e6da-029f-420b-a7b7-64fcd87d1fee None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:23.542827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32e6da-029f-420b-a7b7-64fcd87d1fee None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-5a32e6da-029f-420b-a7b7-64fcd87d1fee None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:23.551414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:23.552701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:23.553650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 657/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 12:41:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:23.559556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f43de3-1be0-4cdc-a089-a74609bc8d84 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:23.655717 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f43de3-1be0-4cdc-a089-a74609bc8d84 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:23.689340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d7950-e504-45c9-be4e-7abf0eb97348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:23.689872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d7950-e504-45c9-be4e-7abf0eb97348 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:23.690149 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d7950-e504-45c9-be4e-7abf0eb97348 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:23.690419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d7950-e504-45c9-be4e-7abf0eb97348 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:23.691098 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d7950-e504-45c9-be4e-7abf0eb97348 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:23.691397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d7950-e504-45c9-be4e-7abf0eb97348 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:23.692740 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d7950-e504-45c9-be4e-7abf0eb97348 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:23.697380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 634/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Fri Sep 9 12:41:23 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:41:23.703318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32e6da-029f-420b-a7b7-64fcd87d1fee None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:23.803570 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32e6da-029f-420b-a7b7-64fcd87d1fee None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:23.835609 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a5bc0-a86c-46ea-84e7-798be7ccb1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:23.836239 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a5bc0-a86c-46ea-84e7-798be7ccb1cb None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:23.836513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a5bc0-a86c-46ea-84e7-798be7ccb1cb None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:23.836780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a5bc0-a86c-46ea-84e7-798be7ccb1cb None admin] PATH_INFO: `/v3/services` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:23.838003 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a5bc0-a86c-46ea-84e7-798be7ccb1cb None admin] RBAC: Adding query filter params (type=placement) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:23.838421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a5bc0-a86c-46ea-84e7-798be7ccb1cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:23.839955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a5bc0-a86c-46ea-84e7-798be7ccb1cb None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:23.844349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 658/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Fri Sep 9 12:41:23 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:26.048290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7d66c-85cb-4001-be5e-b9dc0755e9c7 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:26.048860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7d66c-85cb-4001-be5e-b9dc0755e9c7 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:26.048860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7d66c-85cb-4001-be5e-b9dc0755e9c7 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:26.050149 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 635/1293] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:41:26.055537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f5e1c-d5be-476e-a9dc-0443c8879b56 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:26.055970 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f5e1c-d5be-476e-a9dc-0443c8879b56 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:26.056096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f5e1c-d5be-476e-a9dc-0443c8879b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:26.075103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-352f5e1c-d5be-476e-a9dc-0443c8879b56 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:41:26.076379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-352f5e1c-d5be-476e-a9dc-0443c8879b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:41:26.105121 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:26.146396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 659/1294] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:41:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:41:26.417963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7d66c-85cb-4001-be5e-b9dc0755e9c7 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:26.448466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:26.516559 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7d66c-85cb-4001-be5e-b9dc0755e9c7 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:26.551172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7629b2-1f72-406a-b2fb-31bcf8d29814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:26.551771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7629b2-1f72-406a-b2fb-31bcf8d29814 None admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:26.552051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7629b2-1f72-406a-b2fb-31bcf8d29814 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:26.552323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7629b2-1f72-406a-b2fb-31bcf8d29814 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:26.552979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7629b2-1f72-406a-b2fb-31bcf8d29814 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:26.554336 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7629b2-1f72-406a-b2fb-31bcf8d29814 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:26.637840 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 636/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Fri Sep 9 12:41:26 2022] POST /identity/v3/services => generated 246 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:41:28.835911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eb293-f54d-4b93-92e7-227fa7e486c6 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:28.836462 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eb293-f54d-4b93-92e7-227fa7e486c6 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:28.836462 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eb293-f54d-4b93-92e7-227fa7e486c6 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:28.838080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 660/1296] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:41:28.843379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec612758-98a4-4513-b2b7-ed0e9e943024 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:28.843826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec612758-98a4-4513-b2b7-ed0e9e943024 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:28.843918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec612758-98a4-4513-b2b7-ed0e9e943024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:28.862564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ec612758-98a4-4513-b2b7-ed0e9e943024 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:41:28.863758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ec612758-98a4-4513-b2b7-ed0e9e943024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:41:28.953780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 637/1297] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:41:28 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:41:29.219992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700eb293-f54d-4b93-92e7-227fa7e486c6 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:29.327902 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700eb293-f54d-4b93-92e7-227fa7e486c6 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:29.361045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8abe30-f709-4186-9e3c-1a3b3403c38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:29.361749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8abe30-f709-4186-9e3c-1a3b3403c38e None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:29.362022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8abe30-f709-4186-9e3c-1a3b3403c38e None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:29.362293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8abe30-f709-4186-9e3c-1a3b3403c38e None admin] PATH_INFO: `/v3/services/placement` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:29.362899 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8abe30-f709-4186-9e3c-1a3b3403c38e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:29.364246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8abe30-f709-4186-9e3c-1a3b3403c38e None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-2a8abe30-f709-4186-9e3c-1a3b3403c38e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:29.373796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:29.375067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:29.375998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 661/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 12:41:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:29.380892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec612758-98a4-4513-b2b7-ed0e9e943024 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:29.398799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:29.479552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec612758-98a4-4513-b2b7-ed0e9e943024 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:29.513978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7402ca2-fceb-405c-b43a-5444b77e2972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:29.514602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7402ca2-fceb-405c-b43a-5444b77e2972 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:29.514882 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7402ca2-fceb-405c-b43a-5444b77e2972 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:29.515150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7402ca2-fceb-405c-b43a-5444b77e2972 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:29.515758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7402ca2-fceb-405c-b43a-5444b77e2972 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:29.516052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7402ca2-fceb-405c-b43a-5444b77e2972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:29.517404 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7402ca2-fceb-405c-b43a-5444b77e2972 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:29.522262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 638/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Fri Sep 9 12:41:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:29.528910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8abe30-f709-4186-9e3c-1a3b3403c38e None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:29.560652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:29.635709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8abe30-f709-4186-9e3c-1a3b3403c38e None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:29.669644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f2e30-55ef-4e88-929f-b7a9a63f0bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:29.670242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f2e30-55ef-4e88-929f-b7a9a63f0bad None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:29.670552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f2e30-55ef-4e88-929f-b7a9a63f0bad None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:29.670818 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f2e30-55ef-4e88-929f-b7a9a63f0bad None admin] PATH_INFO: `/v3/endpoints` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:29.671510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f2e30-55ef-4e88-929f-b7a9a63f0bad None admin] RBAC: Adding query filter params (interface=public, service_id=51d23fcbee994cbca59da1833efbcb01, region_id=RegionOne) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:29.671815 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f2e30-55ef-4e88-929f-b7a9a63f0bad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:29.673167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f2e30-55ef-4e88-929f-b7a9a63f0bad None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:29.678004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 662/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Fri Sep 9 12:41:29 2022] GET /identity/v3/endpoints?service_id=51d23fcbee994cbca59da1833efbcb01&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:29.683960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7402ca2-fceb-405c-b43a-5444b77e2972 None admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:29.765328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:29.783229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7402ca2-fceb-405c-b43a-5444b77e2972 None admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:29.816433 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d199f4-c976-46a7-8299-df9926089e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:29.817040 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d199f4-c976-46a7-8299-df9926089e54 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:29.817320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d199f4-c976-46a7-8299-df9926089e54 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:29.817592 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d199f4-c976-46a7-8299-df9926089e54 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:29.818183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d199f4-c976-46a7-8299-df9926089e54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:29.819566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d199f4-c976-46a7-8299-df9926089e54 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:29.827041 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 639/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Fri Sep 9 12:41:29 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:41:31.967389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26cf2e-4e80-4d4f-b0cc-36776bce720b None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:31.967941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26cf2e-4e80-4d4f-b0cc-36776bce720b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:31.967941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26cf2e-4e80-4d4f-b0cc-36776bce720b None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:31.969259 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 663/1302] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:41:31.974370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1c5bf-0598-4430-aad7-3af8241f3f5e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:31.979229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1c5bf-0598-4430-aad7-3af8241f3f5e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:31.979229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1c5bf-0598-4430-aad7-3af8241f3f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:31.993245 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-0be1c5bf-0598-4430-aad7-3af8241f3f5e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:41:31.994600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0be1c5bf-0598-4430-aad7-3af8241f3f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:41:32.059987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 640/1303] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:41:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:41:32.326162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26cf2e-4e80-4d4f-b0cc-36776bce720b None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:32.417874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:32.424370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26cf2e-4e80-4d4f-b0cc-36776bce720b None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:32.455864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e98ad-d7ca-4b80-89fb-8746742e6a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:32.456459 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e98ad-d7ca-4b80-89fb-8746742e6a30 None admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:32.456730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e98ad-d7ca-4b80-89fb-8746742e6a30 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:32.456994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e98ad-d7ca-4b80-89fb-8746742e6a30 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:32.457562 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e98ad-d7ca-4b80-89fb-8746742e6a30 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:32.458923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e98ad-d7ca-4b80-89fb-8746742e6a30 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application [None req-604e98ad-d7ca-4b80-89fb-8746742e6a30 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:41:32.467999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:41:32.469242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application with Lock( Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: ERROR keystone.server.flask.application Sep 09 12:41:32.470298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 664/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Fri Sep 9 12:41:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:41:32.475570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1c5bf-0598-4430-aad7-3af8241f3f5e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:32.551556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:32.574607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1c5bf-0598-4430-aad7-3af8241f3f5e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:32.614675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f63b3-bd2c-40f5-a11e-b29fffda2068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:32.615012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f63b3-bd2c-40f5-a11e-b29fffda2068 None admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:32.615297 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f63b3-bd2c-40f5-a11e-b29fffda2068 None admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:32.615573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f63b3-bd2c-40f5-a11e-b29fffda2068 None admin] PATH_INFO: `/v3/services` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:32.616198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f63b3-bd2c-40f5-a11e-b29fffda2068 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:41:32.616565 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f63b3-bd2c-40f5-a11e-b29fffda2068 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:32.617939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f63b3-bd2c-40f5-a11e-b29fffda2068 None admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:32.623426 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 641/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Fri Sep 9 12:41:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:41:32.629208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e98ad-d7ca-4b80-89fb-8746742e6a30 None admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:41:32.725124 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e98ad-d7ca-4b80-89fb-8746742e6a30 None admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:41:32.757961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2af58-7c09-46d7-976a-0dec96cdf167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:41:32.758585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2af58-7c09-46d7-976a-0dec96cdf167 None admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:41:32.758867 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2af58-7c09-46d7-976a-0dec96cdf167 None admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:41:32.759681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2af58-7c09-46d7-976a-0dec96cdf167 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:41:32.760570 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2af58-7c09-46d7-976a-0dec96cdf167 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:41:32.761944 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2af58-7c09-46d7-976a-0dec96cdf167 None admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:41:32.831014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 665/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:41:32 2022] POST /identity/v3/endpoints => generated 331 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:41:42.620347 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=30 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:41:42.827065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:36.523701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d576d-add8-4d0a-9d4b-9764c3ea8745 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:36.524168 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d576d-add8-4d0a-9d4b-9764c3ea8745 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:36.524379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d576d-add8-4d0a-9d4b-9764c3ea8745 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:36.526551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 642/1307] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:47:36.531674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d7c46-15be-4422-9264-ff88f376ae8b None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:36.532101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d7c46-15be-4422-9264-ff88f376ae8b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:36.532301 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d7c46-15be-4422-9264-ff88f376ae8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:36.644404 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-430d7c46-15be-4422-9264-ff88f376ae8b None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:47:36.645623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-430d7c46-15be-4422-9264-ff88f376ae8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:36.767613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 666/1308] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:47:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:47:37.062407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89e49f-09f0-4a0b-b0c4-a44b1fde0d96 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:37.062778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89e49f-09f0-4a0b-b0c4-a44b1fde0d96 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:37.063003 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89e49f-09f0-4a0b-b0c4-a44b1fde0d96 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:37.064438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 643/1309] 10.222.0.46 () {54 vars in 965 bytes} [Fri Sep 9 12:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:47:37.071699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59dfd5-06f9-40f9-a681-e98dc7e2274e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:37.072136 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59dfd5-06f9-40f9-a681-e98dc7e2274e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:37.072509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59dfd5-06f9-40f9-a681-e98dc7e2274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:37.168563 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-6a59dfd5-06f9-40f9-a681-e98dc7e2274e None None] MFA Rules not processed for user `e46bd9d49bf64013ae58770b0284d81e`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:47:37.170002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6a59dfd5-06f9-40f9-a681-e98dc7e2274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:37.193877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:37.256164 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 667/1310] 10.222.0.46 () {58 vars in 1062 bytes} [Fri Sep 9 12:47:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:47:37.265515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89e49f-09f0-4a0b-b0c4-a44b1fde0d96 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:47:37.267379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ed89e49f-09f0-4a0b-b0c4-a44b1fde0d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:37.385013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89e49f-09f0-4a0b-b0c4-a44b1fde0d96 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:47:37.468865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f6b23-57b2-4aa1-8063-f4afa756b04d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:47:37.469477 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f6b23-57b2-4aa1-8063-f4afa756b04d service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:37.469753 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f6b23-57b2-4aa1-8063-f4afa756b04d service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:37.470035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f6b23-57b2-4aa1-8063-f4afa756b04d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:37.472439 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6b2f6b23-57b2-4aa1-8063-f4afa756b04d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:37.531575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:37.587994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f6b23-57b2-4aa1-8063-f4afa756b04d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:47:37.589415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f6b23-57b2-4aa1-8063-f4afa756b04d service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:47:37.608546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 644/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:47:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:47:40.156299 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4fbd98-408d-455b-b937-e997f431a1c1 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:40.156820 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4fbd98-408d-455b-b937-e997f431a1c1 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:40.156820 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4fbd98-408d-455b-b937-e997f431a1c1 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:40.158102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 668/1312] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:47:40.163491 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec665d8a-5bbe-4262-9448-6a55d0919ef9 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:40.163922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec665d8a-5bbe-4262-9448-6a55d0919ef9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:40.164048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec665d8a-5bbe-4262-9448-6a55d0919ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:40.235354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ec665d8a-5bbe-4262-9448-6a55d0919ef9 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:47:40.236543 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ec665d8a-5bbe-4262-9448-6a55d0919ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:40.335208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 645/1313] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:47:40.621906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4fbd98-408d-455b-b937-e997f431a1c1 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:47:40.623569 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7c4fbd98-408d-455b-b937-e997f431a1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:40.729979 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4fbd98-408d-455b-b937-e997f431a1c1 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:47:40.801451 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:40.803717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa25471-75be-424c-8792-503da923f916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:47:40.804314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa25471-75be-424c-8792-503da923f916 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:40.804586 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa25471-75be-424c-8792-503da923f916 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:40.804854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa25471-75be-424c-8792-503da923f916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:40.806430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-dfa25471-75be-424c-8792-503da923f916 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:40.915223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa25471-75be-424c-8792-503da923f916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:47:40.916556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa25471-75be-424c-8792-503da923f916 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:47:40.935241 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 669/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:47:43.660610 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63c3b5-5ae4-4234-90f4-fdd039f99fd1 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:43.661177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63c3b5-5ae4-4234-90f4-fdd039f99fd1 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:43.661177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63c3b5-5ae4-4234-90f4-fdd039f99fd1 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:43.662583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 646/1315] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:47:43.667774 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6c990-17b0-49ca-9cdc-62c83417081a None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:43.668195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6c990-17b0-49ca-9cdc-62c83417081a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:43.668289 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6c990-17b0-49ca-9cdc-62c83417081a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:43.735046 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-27b6c990-17b0-49ca-9cdc-62c83417081a None None] MFA Rules not processed for user `4694cfe2faf74f7085cc07187b0f047f`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:47:43.736294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-27b6c990-17b0-49ca-9cdc-62c83417081a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:43.789005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:43.834164 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 670/1316] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:47:43 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:47:44.131755 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63c3b5-5ae4-4234-90f4-fdd039f99fd1 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:47:44.133288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ad63c3b5-5ae4-4234-90f4-fdd039f99fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:44.143271 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:44.236357 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63c3b5-5ae4-4234-90f4-fdd039f99fd1 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:47:44.309604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddf429-1141-42e5-aeb3-7bc02a62a759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:47:44.310213 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddf429-1141-42e5-aeb3-7bc02a62a759 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:44.310535 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddf429-1141-42e5-aeb3-7bc02a62a759 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:44.310812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddf429-1141-42e5-aeb3-7bc02a62a759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:44.312397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e0ddf429-1141-42e5-aeb3-7bc02a62a759 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:44.375298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:44.420711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddf429-1141-42e5-aeb3-7bc02a62a759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:47:44.422133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddf429-1141-42e5-aeb3-7bc02a62a759 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:47:44.440846 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 647/1317] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:47:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:47:47.297489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e139e-cdca-4182-9c93-80667fd9a856 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:47.298049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e139e-cdca-4182-9c93-80667fd9a856 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:47.298049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e139e-cdca-4182-9c93-80667fd9a856 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:47.300376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 671/1318] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:47:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:47:47.305754 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446ca99-edcb-42a5-931f-d5d2ccac26cd None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:47.306213 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446ca99-edcb-42a5-931f-d5d2ccac26cd None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:47.306325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446ca99-edcb-42a5-931f-d5d2ccac26cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:47.371046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-0446ca99-edcb-42a5-931f-d5d2ccac26cd None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:47:47.372236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0446ca99-edcb-42a5-931f-d5d2ccac26cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:47.466844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 648/1319] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:47:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:47:47.782653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8ab6-e565-4b8b-89cc-019a54653c25 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:47.782933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8ab6-e565-4b8b-89cc-019a54653c25 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:47.783205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8ab6-e565-4b8b-89cc-019a54653c25 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:47.784837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 672/1320] 10.222.0.46 () {54 vars in 965 bytes} [Fri Sep 9 12:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:47:47.790700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cd25b-a868-4166-a1a2-2c7e08eff323 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:47.790991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cd25b-a868-4166-a1a2-2c7e08eff323 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:47.791267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cd25b-a868-4166-a1a2-2c7e08eff323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:47.813260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:47.866545 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-165cd25b-a868-4166-a1a2-2c7e08eff323 None None] MFA Rules not processed for user `e46bd9d49bf64013ae58770b0284d81e`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:47:47.867731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-165cd25b-a868-4166-a1a2-2c7e08eff323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:47.942005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 649/1321] 10.222.0.46 () {58 vars in 1062 bytes} [Fri Sep 9 12:47:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:47:47.951318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb8ab6-e565-4b8b-89cc-019a54653c25 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:47:47.952816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fdbb8ab6-e565-4b8b-89cc-019a54653c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:48.051693 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb8ab6-e565-4b8b-89cc-019a54653c25 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:47:48.122532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a641b-e081-42ba-8d07-7b595557d175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:47:48.123301 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a641b-e081-42ba-8d07-7b595557d175 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:48.123410 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a641b-e081-42ba-8d07-7b595557d175 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:48.123643 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a641b-e081-42ba-8d07-7b595557d175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:48.125210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-314a641b-e081-42ba-8d07-7b595557d175 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:48.132046 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:48.231173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a641b-e081-42ba-8d07-7b595557d175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:47:48.232571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a641b-e081-42ba-8d07-7b595557d175 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:47:48.250432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 673/1322] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:47:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:47:51.777660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445e688-8199-46a4-a2cf-881468cfd381 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:51.778229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445e688-8199-46a4-a2cf-881468cfd381 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:51.778229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445e688-8199-46a4-a2cf-881468cfd381 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:51.779576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 650/1323] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:47:51.785352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab513107-d8bb-4240-95bb-6510ebd3940a None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:51.785796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab513107-d8bb-4240-95bb-6510ebd3940a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:51.785912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab513107-d8bb-4240-95bb-6510ebd3940a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:51.856835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-ab513107-d8bb-4240-95bb-6510ebd3940a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:47:51.858026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ab513107-d8bb-4240-95bb-6510ebd3940a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:51.917567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:51.960790 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 674/1324] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:47:52.246207 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445e688-8199-46a4-a2cf-881468cfd381 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:47:52.247773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7445e688-8199-46a4-a2cf-881468cfd381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:52.354209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445e688-8199-46a4-a2cf-881468cfd381 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:47:52.367893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:52.429476 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4207fd-ad70-4c1a-9abc-4193ca91114f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:47:52.430077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4207fd-ad70-4c1a-9abc-4193ca91114f service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:52.430353 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4207fd-ad70-4c1a-9abc-4193ca91114f service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:52.430656 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4207fd-ad70-4c1a-9abc-4193ca91114f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:52.432227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ee4207fd-ad70-4c1a-9abc-4193ca91114f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:52.540293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4207fd-ad70-4c1a-9abc-4193ca91114f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:47:52.541714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4207fd-ad70-4c1a-9abc-4193ca91114f service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:47:52.559909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 651/1325] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:47:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:47:54.786105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528a718-c1e2-4108-b614-ad7f7c2ecc16 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:54.786720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528a718-c1e2-4108-b614-ad7f7c2ecc16 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:54.786720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528a718-c1e2-4108-b614-ad7f7c2ecc16 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:54.788066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 675/1326] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:47:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:47:54.794176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440591af-c9bd-4aef-ab63-5611effa362e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:54.794747 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440591af-c9bd-4aef-ab63-5611effa362e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:54.794747 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440591af-c9bd-4aef-ab63-5611effa362e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:54.845705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:54.865180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-440591af-c9bd-4aef-ab63-5611effa362e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:47:54.866378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-440591af-c9bd-4aef-ab63-5611effa362e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:54.964491 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 652/1327] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:47:55.248442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528a718-c1e2-4108-b614-ad7f7c2ecc16 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:47:55.250011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2528a718-c1e2-4108-b614-ad7f7c2ecc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:55.354552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528a718-c1e2-4108-b614-ad7f7c2ecc16 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:47:55.456994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0cdbe-b551-44e0-a6bc-91b949ea8913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:47:55.457600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0cdbe-b551-44e0-a6bc-91b949ea8913 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:55.457875 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0cdbe-b551-44e0-a6bc-91b949ea8913 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:55.458144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0cdbe-b551-44e0-a6bc-91b949ea8913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:55.459776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9ba0cdbe-b551-44e0-a6bc-91b949ea8913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:55.477961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:55.573972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0cdbe-b551-44e0-a6bc-91b949ea8913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:47:55.575388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0cdbe-b551-44e0-a6bc-91b949ea8913 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:47:55.593276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 676/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:47:57.829064 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e6ffa-caf2-438e-92fe-16973cc2f7b4 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:57.829615 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e6ffa-caf2-438e-92fe-16973cc2f7b4 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:57.829615 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e6ffa-caf2-438e-92fe-16973cc2f7b4 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:57.831069 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 653/1329] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:47:57.836258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090693d1-8fcc-4d23-905b-722f0bf85127 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:57.836724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090693d1-8fcc-4d23-905b-722f0bf85127 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:57.836816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090693d1-8fcc-4d23-905b-722f0bf85127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:57.904365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-090693d1-8fcc-4d23-905b-722f0bf85127 None None] MFA Rules not processed for user `4694cfe2faf74f7085cc07187b0f047f`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:47:57.905575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-090693d1-8fcc-4d23-905b-722f0bf85127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:57.975158 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:58.008398 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 677/1330] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:47:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:47:58.294579 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e6ffa-caf2-438e-92fe-16973cc2f7b4 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:47:58.296223 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e11e6ffa-caf2-438e-92fe-16973cc2f7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:58.416495 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e6ffa-caf2-438e-92fe-16973cc2f7b4 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:47:58.439922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:47:58.497314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849aa4e-f0f6-484f-aba7-17a2f0d9b386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:47:58.497848 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849aa4e-f0f6-484f-aba7-17a2f0d9b386 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:47:58.498127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849aa4e-f0f6-484f-aba7-17a2f0d9b386 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:47:58.498398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849aa4e-f0f6-484f-aba7-17a2f0d9b386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:47:58.500027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f849aa4e-f0f6-484f-aba7-17a2f0d9b386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:47:58.612973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849aa4e-f0f6-484f-aba7-17a2f0d9b386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:47:58.614386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849aa4e-f0f6-484f-aba7-17a2f0d9b386 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:47:58.632624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 654/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:47:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:01.247555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed4d30-be78-4b9c-b36a-383653782265 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:01.248127 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed4d30-be78-4b9c-b36a-383653782265 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:01.248127 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed4d30-be78-4b9c-b36a-383653782265 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:01.249429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 678/1332] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:01.254745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf390ce-6298-4406-b9b1-c78bf3727c4a None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:01.255207 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf390ce-6298-4406-b9b1-c78bf3727c4a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:01.255307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf390ce-6298-4406-b9b1-c78bf3727c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:01.318271 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:01.320780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-aaf390ce-6298-4406-b9b1-c78bf3727c4a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:01.323820 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-aaf390ce-6298-4406-b9b1-c78bf3727c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:01.416858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 655/1333] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:01.708678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed4d30-be78-4b9c-b36a-383653782265 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:01.710220 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-39ed4d30-be78-4b9c-b36a-383653782265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:01.810962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed4d30-be78-4b9c-b36a-383653782265 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:01.884140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ea047-fab9-4749-8265-b4994489851c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:01.884724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ea047-fab9-4749-8265-b4994489851c service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:01.884998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ea047-fab9-4749-8265-b4994489851c service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:01.885262 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ea047-fab9-4749-8265-b4994489851c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:01.886862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-cc6ea047-fab9-4749-8265-b4994489851c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:01.912730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:01.991528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ea047-fab9-4749-8265-b4994489851c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:01.992924 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ea047-fab9-4749-8265-b4994489851c service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:02.011072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 679/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:04.202417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a6761-76cd-4bf0-a309-8bc600c0abfd None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:04.202986 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a6761-76cd-4bf0-a309-8bc600c0abfd None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:04.202986 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a6761-76cd-4bf0-a309-8bc600c0abfd None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:04.204251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 656/1335] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:04.209528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c2aa2-2678-4fc5-bcd0-c2500f366089 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:04.209971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c2aa2-2678-4fc5-bcd0-c2500f366089 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:04.210074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c2aa2-2678-4fc5-bcd0-c2500f366089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:04.274468 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-052c2aa2-2678-4fc5-bcd0-c2500f366089 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:04.275687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-052c2aa2-2678-4fc5-bcd0-c2500f366089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:04.348742 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:04.370333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 680/1336] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:04.663841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a6761-76cd-4bf0-a309-8bc600c0abfd None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:04.665379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-8f3a6761-76cd-4bf0-a309-8bc600c0abfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:04.768748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a6761-76cd-4bf0-a309-8bc600c0abfd None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:04.796276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:04.842949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb232a1-ff6e-4adc-9f7e-45635e2f90ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:04.843550 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb232a1-ff6e-4adc-9f7e-45635e2f90ff service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:04.843830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb232a1-ff6e-4adc-9f7e-45635e2f90ff service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:04.844100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb232a1-ff6e-4adc-9f7e-45635e2f90ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:04.845707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2cb232a1-ff6e-4adc-9f7e-45635e2f90ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:04.952704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb232a1-ff6e-4adc-9f7e-45635e2f90ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:04.954043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb232a1-ff6e-4adc-9f7e-45635e2f90ff service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:04.971624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 657/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:07.730331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddfbea-ebdf-4934-95ef-83a3b8f82e55 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:07.730933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddfbea-ebdf-4934-95ef-83a3b8f82e55 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:07.730933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddfbea-ebdf-4934-95ef-83a3b8f82e55 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:07.732265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 681/1338] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:07.738037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21da69d-715b-48ed-8db6-b51732f2cd3e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:07.738431 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21da69d-715b-48ed-8db6-b51732f2cd3e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:07.738602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21da69d-715b-48ed-8db6-b51732f2cd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:07.804377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-e21da69d-715b-48ed-8db6-b51732f2cd3e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:07.805562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e21da69d-715b-48ed-8db6-b51732f2cd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:07.813658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:07.903133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 658/1339] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:08.185133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddfbea-ebdf-4934-95ef-83a3b8f82e55 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:08.186685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-27ddfbea-ebdf-4934-95ef-83a3b8f82e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:08.294238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddfbea-ebdf-4934-95ef-83a3b8f82e55 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:08.374018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea46a1-c03a-4b5e-8372-b03821f44632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:08.374562 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea46a1-c03a-4b5e-8372-b03821f44632 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:08.374844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea46a1-c03a-4b5e-8372-b03821f44632 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:08.375116 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea46a1-c03a-4b5e-8372-b03821f44632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:08.379372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-feea46a1-c03a-4b5e-8372-b03821f44632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:08.417273 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:08.490321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea46a1-c03a-4b5e-8372-b03821f44632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:08.491762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea46a1-c03a-4b5e-8372-b03821f44632 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:08.509955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 682/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:12.881988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d4bc-d06a-4d04-ac4c-415fc2a779f9 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:12.882526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d4bc-d06a-4d04-ac4c-415fc2a779f9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:12.882623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d4bc-d06a-4d04-ac4c-415fc2a779f9 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:12.883989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 659/1341] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:12.889354 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6666508-fdab-4d66-821d-d03a6719c84a None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:12.889770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6666508-fdab-4d66-821d-d03a6719c84a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:12.889883 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6666508-fdab-4d66-821d-d03a6719c84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:12.957231 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b6666508-fdab-4d66-821d-d03a6719c84a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:12.958421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b6666508-fdab-4d66-821d-d03a6719c84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:13.044824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:13.056316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 683/1342] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:13.340402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4d4bc-d06a-4d04-ac4c-415fc2a779f9 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:13.341949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-67d4d4bc-d06a-4d04-ac4c-415fc2a779f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:13.445465 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4d4bc-d06a-4d04-ac4c-415fc2a779f9 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:13.483691 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:13.523504 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b4e73-70d6-46f4-84b9-e945a0218196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:13.524111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b4e73-70d6-46f4-84b9-e945a0218196 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:13.524389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b4e73-70d6-46f4-84b9-e945a0218196 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:13.524659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b4e73-70d6-46f4-84b9-e945a0218196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:13.526245 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-237b4e73-70d6-46f4-84b9-e945a0218196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:13.634536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b4e73-70d6-46f4-84b9-e945a0218196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:13.635984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b4e73-70d6-46f4-84b9-e945a0218196 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:13.654916 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 660/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:16.695214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405d155-5b45-4b2f-8447-37507a1c8260 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:16.695754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405d155-5b45-4b2f-8447-37507a1c8260 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:16.695754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405d155-5b45-4b2f-8447-37507a1c8260 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:16.697262 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 684/1344] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:16.702672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5330eb-9e87-42c5-8c6d-c70c3d07052f None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:16.703302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5330eb-9e87-42c5-8c6d-c70c3d07052f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:16.703302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5330eb-9e87-42c5-8c6d-c70c3d07052f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:16.770095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-7f5330eb-9e87-42c5-8c6d-c70c3d07052f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:16.771332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7f5330eb-9e87-42c5-8c6d-c70c3d07052f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:16.789665 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:16.872179 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 661/1345] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:17.187091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405d155-5b45-4b2f-8447-37507a1c8260 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:17.188661 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2405d155-5b45-4b2f-8447-37507a1c8260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:17.310975 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405d155-5b45-4b2f-8447-37507a1c8260 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:17.397158 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee4ff5-1817-4d66-baa5-71b7fd1c49c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:17.397681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee4ff5-1817-4d66-baa5-71b7fd1c49c1 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:17.397956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee4ff5-1817-4d66-baa5-71b7fd1c49c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:17.398227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee4ff5-1817-4d66-baa5-71b7fd1c49c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:17.400737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5eee4ff5-1817-4d66-baa5-71b7fd1c49c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:17.451660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:17.551129 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee4ff5-1817-4d66-baa5-71b7fd1c49c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:17.552567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee4ff5-1817-4d66-baa5-71b7fd1c49c1 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:17.573834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 685/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:22.153909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e00c5c-0edb-4035-8054-2aef31978e1f None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:22.153909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e00c5c-0edb-4035-8054-2aef31978e1f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:22.153909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e00c5c-0edb-4035-8054-2aef31978e1f None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:22.154639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 662/1347] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:22.160281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd19e4-c01b-4dce-8022-07e9eff2df3c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:22.160763 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd19e4-c01b-4dce-8022-07e9eff2df3c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:22.160859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd19e4-c01b-4dce-8022-07e9eff2df3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:22.247704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-1edd19e4-c01b-4dce-8022-07e9eff2df3c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:22.247704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1edd19e4-c01b-4dce-8022-07e9eff2df3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:22.342213 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:22.344544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 686/1348] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:48:22.635075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e00c5c-0edb-4035-8054-2aef31978e1f None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:22.636659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a0e00c5c-0edb-4035-8054-2aef31978e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:22.739762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e00c5c-0edb-4035-8054-2aef31978e1f None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:22.813338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:22.839889 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eaa561-c8da-4650-b090-bde53ac5c6b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:22.840488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaa561-c8da-4650-b090-bde53ac5c6b9 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:22.840766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaa561-c8da-4650-b090-bde53ac5c6b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:22.841035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaa561-c8da-4650-b090-bde53ac5c6b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:22.842642 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-85eaa561-c8da-4650-b090-bde53ac5c6b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:22.953893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eaa561-c8da-4650-b090-bde53ac5c6b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:22.955940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eaa561-c8da-4650-b090-bde53ac5c6b9 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:22.973622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 663/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:27.909713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acb36a-a137-498d-8859-427ca9e55204 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:27.910520 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acb36a-a137-498d-8859-427ca9e55204 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:27.910964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acb36a-a137-498d-8859-427ca9e55204 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:27.913520 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 687/1350] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 12:48:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:27.917538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a3c16-f21c-4c0e-a518-641328e3bc31 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:27.917971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a3c16-f21c-4c0e-a518-641328e3bc31 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:27.918096 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a3c16-f21c-4c0e-a518-641328e3bc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:28.016784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-617a3c16-f21c-4c0e-a518-641328e3bc31 None None] MFA Rules not processed for user `33e177d86450489cacc86578ba26509d`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:28.018665 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-617a3c16-f21c-4c0e-a518-641328e3bc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:28.050329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:28.190288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 664/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 12:48:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:32.361322 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b1749-3ea4-4ea6-a6bf-b5c796344a5a None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:32.361995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b1749-3ea4-4ea6-a6bf-b5c796344a5a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:32.361995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b1749-3ea4-4ea6-a6bf-b5c796344a5a None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:32.364068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 688/1352] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 12:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:32.369080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa73b26-6e9f-401e-b432-270d05ce3080 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:32.369571 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa73b26-6e9f-401e-b432-270d05ce3080 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:32.369681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa73b26-6e9f-401e-b432-270d05ce3080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:32.456533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-eaa73b26-6e9f-401e-b432-270d05ce3080 None None] MFA Rules not processed for user `33e177d86450489cacc86578ba26509d`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:32.458136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-eaa73b26-6e9f-401e-b432-270d05ce3080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:32.536135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:32.575588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 665/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 12:48:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:34.032886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf924a7-b57a-48ff-a30c-d6807aaa6ea3 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:34.033470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf924a7-b57a-48ff-a30c-d6807aaa6ea3 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:34.033470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf924a7-b57a-48ff-a30c-d6807aaa6ea3 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:34.035156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 689/1354] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 12:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:34.039808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8e8b4-cf25-4c16-a3a2-bae03038f5c5 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:34.040256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8e8b4-cf25-4c16-a3a2-bae03038f5c5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:34.040489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8e8b4-cf25-4c16-a3a2-bae03038f5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:34.118731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-25f8e8b4-cf25-4c16-a3a2-bae03038f5c5 None None] MFA Rules not processed for user `33e177d86450489cacc86578ba26509d`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:34.120216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-25f8e8b4-cf25-4c16-a3a2-bae03038f5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:34.237082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 666/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 12:48:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:48:37.705131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149d971-21fe-4b86-8dd0-88c57159a2e0 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:37.705676 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149d971-21fe-4b86-8dd0-88c57159a2e0 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:37.705773 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149d971-21fe-4b86-8dd0-88c57159a2e0 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:37.707114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 690/1356] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:37.712806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d48ca1-e949-478c-8ff4-3d5d22de7026 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:37.713270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d48ca1-e949-478c-8ff4-3d5d22de7026 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:37.713376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d48ca1-e949-478c-8ff4-3d5d22de7026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:37.755707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:37.789330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-44d48ca1-e949-478c-8ff4-3d5d22de7026 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:37.790443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-44d48ca1-e949-478c-8ff4-3d5d22de7026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:37.893744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 667/1357] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:38.197930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946008f-2a3c-4649-8a5f-7c5db01a77db None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:38.198614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946008f-2a3c-4649-8a5f-7c5db01a77db None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:38.199206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946008f-2a3c-4649-8a5f-7c5db01a77db None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:38.201845 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 691/1358] 10.222.0.46 () {54 vars in 962 bytes} [Fri Sep 9 12:48:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:38.215082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47deca-adb3-4b90-b754-45949d69b8e6 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:38.215422 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47deca-adb3-4b90-b754-45949d69b8e6 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:38.215711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47deca-adb3-4b90-b754-45949d69b8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:38.325975 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-7d47deca-adb3-4b90-b754-45949d69b8e6 None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:38.327272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7d47deca-adb3-4b90-b754-45949d69b8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:38.381730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:38.454118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 668/1359] 10.222.0.46 () {58 vars in 1059 bytes} [Fri Sep 9 12:48:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:38.463176 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946008f-2a3c-4649-8a5f-7c5db01a77db None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:38.464857 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2946008f-2a3c-4649-8a5f-7c5db01a77db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:38.627684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946008f-2a3c-4649-8a5f-7c5db01a77db None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:38.706076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:38.750184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45483dbb-9e98-4322-acc7-a4a5c98ff56e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:38.750820 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45483dbb-9e98-4322-acc7-a4a5c98ff56e service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:38.751102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45483dbb-9e98-4322-acc7-a4a5c98ff56e service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:38.751560 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45483dbb-9e98-4322-acc7-a4a5c98ff56e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:38.753188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-45483dbb-9e98-4322-acc7-a4a5c98ff56e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:38.856018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45483dbb-9e98-4322-acc7-a4a5c98ff56e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:38.857671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45483dbb-9e98-4322-acc7-a4a5c98ff56e service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:38.879084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 692/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:41.276118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55329c60-6288-4807-b227-3ecf0d9d7985 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:41.276663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55329c60-6288-4807-b227-3ecf0d9d7985 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:41.276829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55329c60-6288-4807-b227-3ecf0d9d7985 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:41.278264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 669/1361] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:41.284127 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd1715-33fc-4eac-afa7-7162ce93e45c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:41.284589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd1715-33fc-4eac-afa7-7162ce93e45c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:41.284684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd1715-33fc-4eac-afa7-7162ce93e45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:41.364017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d6dd1715-33fc-4eac-afa7-7162ce93e45c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:41.366161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d6dd1715-33fc-4eac-afa7-7162ce93e45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:41.431746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:41.502221 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 693/1362] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:41.808401 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55329c60-6288-4807-b227-3ecf0d9d7985 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:41.810175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-55329c60-6288-4807-b227-3ecf0d9d7985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:41.989723 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55329c60-6288-4807-b227-3ecf0d9d7985 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:42.002235 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:42.132902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433fbf4-4d78-48ae-b119-43a86662c253 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:42.133509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433fbf4-4d78-48ae-b119-43a86662c253 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:42.133789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433fbf4-4d78-48ae-b119-43a86662c253 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:42.134061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433fbf4-4d78-48ae-b119-43a86662c253 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:42.135689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0433fbf4-4d78-48ae-b119-43a86662c253 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:42.248336 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433fbf4-4d78-48ae-b119-43a86662c253 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:42.248336 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433fbf4-4d78-48ae-b119-43a86662c253 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:42.264442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 670/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:44.869757 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412504d-890f-4ad7-91ab-5a8a15aa1994 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:44.870288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412504d-890f-4ad7-91ab-5a8a15aa1994 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:44.870407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412504d-890f-4ad7-91ab-5a8a15aa1994 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:44.871858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 694/1364] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:44.877690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2f504-bb67-40f2-9116-83de88ed5efd None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:44.878564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2f504-bb67-40f2-9116-83de88ed5efd None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:44.879142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2f504-bb67-40f2-9116-83de88ed5efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:44.897677 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:44.962041 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-45f2f504-bb67-40f2-9116-83de88ed5efd None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:44.963806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-45f2f504-bb67-40f2-9116-83de88ed5efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:45.072633 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 671/1365] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:45.380041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230507f-3447-4658-a880-edf86b1e87e1 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:45.380303 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230507f-3447-4658-a880-edf86b1e87e1 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:45.380538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230507f-3447-4658-a880-edf86b1e87e1 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:45.382005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 695/1366] 10.222.0.46 () {54 vars in 962 bytes} [Fri Sep 9 12:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:45.388113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25fa31-e106-4bec-a241-0b1969a94398 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:45.388480 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25fa31-e106-4bec-a241-0b1969a94398 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:45.388796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25fa31-e106-4bec-a241-0b1969a94398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:45.468320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-4c25fa31-e106-4bec-a241-0b1969a94398 None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:45.469521 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-4c25fa31-e106-4bec-a241-0b1969a94398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:45.510430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:45.625033 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 672/1367] 10.222.0.46 () {58 vars in 1059 bytes} [Fri Sep 9 12:48:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:48:45.632316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230507f-3447-4658-a880-edf86b1e87e1 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:45.634307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c230507f-3447-4658-a880-edf86b1e87e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:45.801136 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230507f-3447-4658-a880-edf86b1e87e1 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:45.823510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:45.932060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b7015-f021-4dac-a154-6d851af817a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:45.932696 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b7015-f021-4dac-a154-6d851af817a8 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:45.932935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b7015-f021-4dac-a154-6d851af817a8 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:45.933318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b7015-f021-4dac-a154-6d851af817a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:45.934945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-250b7015-f021-4dac-a154-6d851af817a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:46.039606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b7015-f021-4dac-a154-6d851af817a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:46.041195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b7015-f021-4dac-a154-6d851af817a8 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:46.060773 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 696/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:46.089088 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88d296-7251-4ea8-bd60-822fd305992b None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:46.089590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88d296-7251-4ea8-bd60-822fd305992b None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:46.089702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88d296-7251-4ea8-bd60-822fd305992b None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:46.091141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 673/1369] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:46.096651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527da432-d515-41b4-93b4-23ef715e638c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:46.096913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527da432-d515-41b4-93b4-23ef715e638c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:46.097335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527da432-d515-41b4-93b4-23ef715e638c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:46.132328 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:46.174280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-527da432-d515-41b4-93b4-23ef715e638c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:46.175546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-527da432-d515-41b4-93b4-23ef715e638c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:46.300622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 697/1370] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:48:46.640529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fef4c-04b0-4810-8c8f-acc5edd682b8 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:46.640820 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fef4c-04b0-4810-8c8f-acc5edd682b8 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:46.641096 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fef4c-04b0-4810-8c8f-acc5edd682b8 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:46.643624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 674/1371] 10.222.0.46 () {54 vars in 964 bytes} [Fri Sep 9 12:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:46.648764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584a861-e312-4ffc-bd46-e9aba66e7ccc None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:46.649132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584a861-e312-4ffc-bd46-e9aba66e7ccc None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:46.649480 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584a861-e312-4ffc-bd46-e9aba66e7ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:46.734522 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-3584a861-e312-4ffc-bd46-e9aba66e7ccc None None] MFA Rules not processed for user `002c117a85c1452e9fae80c755c6622f`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:46.736152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-3584a861-e312-4ffc-bd46-e9aba66e7ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:46.785462 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:46.816386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 698/1372] 10.222.0.46 () {58 vars in 1061 bytes} [Fri Sep 9 12:48:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:48:46.824065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fef4c-04b0-4810-8c8f-acc5edd682b8 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:46.825555 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f08fef4c-04b0-4810-8c8f-acc5edd682b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:46.931185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fef4c-04b0-4810-8c8f-acc5edd682b8 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:46.978721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:47.019362 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf58b4-ccd9-426f-868d-a2755da3a524 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:47.019966 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf58b4-ccd9-426f-868d-a2755da3a524 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:47.020309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf58b4-ccd9-426f-868d-a2755da3a524 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:47.020508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf58b4-ccd9-426f-868d-a2755da3a524 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:47.022099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2daf58b4-ccd9-426f-868d-a2755da3a524 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:47.158471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf58b4-ccd9-426f-868d-a2755da3a524 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:47.159941 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf58b4-ccd9-426f-868d-a2755da3a524 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:47.180888 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 675/1373] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:48.426377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04f5d4-24b8-4aa1-994f-eec5ac59721a None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:48.426938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04f5d4-24b8-4aa1-994f-eec5ac59721a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:48.426938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04f5d4-24b8-4aa1-994f-eec5ac59721a None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:48.428221 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 699/1374] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:48.433461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38033a09-dc0e-4b96-b184-b24b972f81c1 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:48.433888 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38033a09-dc0e-4b96-b184-b24b972f81c1 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:48.434003 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38033a09-dc0e-4b96-b184-b24b972f81c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:48.499963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-38033a09-dc0e-4b96-b184-b24b972f81c1 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:48.501296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-38033a09-dc0e-4b96-b184-b24b972f81c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:48.522154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:48.604004 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 676/1375] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:48.896086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04f5d4-24b8-4aa1-994f-eec5ac59721a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:48.897642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6d04f5d4-24b8-4aa1-994f-eec5ac59721a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:49.059485 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04f5d4-24b8-4aa1-994f-eec5ac59721a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:49.112833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:49.178870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e0ed2-dd4c-4f22-9305-f134e792b369 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:49.179470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e0ed2-dd4c-4f22-9305-f134e792b369 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:49.179740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e0ed2-dd4c-4f22-9305-f134e792b369 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:49.180010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e0ed2-dd4c-4f22-9305-f134e792b369 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:49.181587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fe4e0ed2-dd4c-4f22-9305-f134e792b369 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:49.281134 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e0ed2-dd4c-4f22-9305-f134e792b369 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:49.283343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e0ed2-dd4c-4f22-9305-f134e792b369 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:49.305079 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 700/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:51.707141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d909c-14e6-4a45-b40e-d249a359f78c None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:51.707712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d909c-14e6-4a45-b40e-d249a359f78c None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:51.707712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d909c-14e6-4a45-b40e-d249a359f78c None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:51.709172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 677/1377] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:51.714738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca70b7-c8b1-4476-a23c-ba2103f01bab None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:51.715220 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca70b7-c8b1-4476-a23c-ba2103f01bab None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:51.715318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca70b7-c8b1-4476-a23c-ba2103f01bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:51.789104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-70ca70b7-c8b1-4476-a23c-ba2103f01bab None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:51.790319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-70ca70b7-c8b1-4476-a23c-ba2103f01bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:51.797466 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:51.895358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 701/1378] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:52.186075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d909c-14e6-4a45-b40e-d249a359f78c None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:52.187616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-195d909c-14e6-4a45-b40e-d249a359f78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:52.343453 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d909c-14e6-4a45-b40e-d249a359f78c None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:52.347735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:52.484868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9fb83-2434-4ec4-93c6-01e5ceee82d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:52.486142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9fb83-2434-4ec4-93c6-01e5ceee82d8 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:52.486904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9fb83-2434-4ec4-93c6-01e5ceee82d8 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:52.487199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9fb83-2434-4ec4-93c6-01e5ceee82d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:52.489139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-82a9fb83-2434-4ec4-93c6-01e5ceee82d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:52.590358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9fb83-2434-4ec4-93c6-01e5ceee82d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:52.592058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9fb83-2434-4ec4-93c6-01e5ceee82d8 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:52.611343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 678/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:54.993910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f96e4-f59a-40e6-a504-bbf12edb6f40 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:54.994461 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f96e4-f59a-40e6-a504-bbf12edb6f40 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:54.994461 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f96e4-f59a-40e6-a504-bbf12edb6f40 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:54.995631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 702/1380] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:55.001821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fc675-06b3-4e15-aaa5-b5c0886b3082 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:55.002454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fc675-06b3-4e15-aaa5-b5c0886b3082 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:55.002454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fc675-06b3-4e15-aaa5-b5c0886b3082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:55.013800 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:55.078768 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-a78fc675-06b3-4e15-aaa5-b5c0886b3082 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:55.081023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a78fc675-06b3-4e15-aaa5-b5c0886b3082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:55.206410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 679/1381] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:48:55.501276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f96e4-f59a-40e6-a504-bbf12edb6f40 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:55.502871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e25f96e4-f59a-40e6-a504-bbf12edb6f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:55.643877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:55.654241 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f96e4-f59a-40e6-a504-bbf12edb6f40 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:55.769531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae556d-61d0-4adc-be5a-1a778a5880c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:55.770048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae556d-61d0-4adc-be5a-1a778a5880c5 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:55.770322 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae556d-61d0-4adc-be5a-1a778a5880c5 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:55.770624 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae556d-61d0-4adc-be5a-1a778a5880c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:55.772239 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-13ae556d-61d0-4adc-be5a-1a778a5880c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:55.885967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae556d-61d0-4adc-be5a-1a778a5880c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:55.887607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae556d-61d0-4adc-be5a-1a778a5880c5 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:55.898084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:55.906272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 703/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:48:58.795160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d18610-04df-4680-b3bb-528d5b15ee34 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:58.795728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d18610-04df-4680-b3bb-528d5b15ee34 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:58.795728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d18610-04df-4680-b3bb-528d5b15ee34 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:58.797091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 680/1383] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:48:58.803607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ac3c2-2294-4003-bc5b-e40c7ba8cb97 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:58.804038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ac3c2-2294-4003-bc5b-e40c7ba8cb97 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:58.804156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ac3c2-2294-4003-bc5b-e40c7ba8cb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:58.871916 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-618ac3c2-2294-4003-bc5b-e40c7ba8cb97 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:48:58.873243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-618ac3c2-2294-4003-bc5b-e40c7ba8cb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:59.019090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 704/1384] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:48:59.328742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d18610-04df-4680-b3bb-528d5b15ee34 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:48:59.330295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-81d18610-04df-4680-b3bb-528d5b15ee34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:59.422337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:59.488064 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d18610-04df-4680-b3bb-528d5b15ee34 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:48:59.618548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44c1e1-fc1c-4aa5-ab42-910e354f6450 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:48:59.619161 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44c1e1-fc1c-4aa5-ab42-910e354f6450 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:48:59.619436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44c1e1-fc1c-4aa5-ab42-910e354f6450 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:48:59.619700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44c1e1-fc1c-4aa5-ab42-910e354f6450 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:48:59.621326 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7e44c1e1-fc1c-4aa5-ab42-910e354f6450 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:48:59.680031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:48:59.718994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44c1e1-fc1c-4aa5-ab42-910e354f6450 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:48:59.721131 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44c1e1-fc1c-4aa5-ab42-910e354f6450 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:48:59.740661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 681/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:48:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:01.954382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd19090-6b93-475c-b6c7-ba8fd6240d8c None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:01.954885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd19090-6b93-475c-b6c7-ba8fd6240d8c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:01.955002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd19090-6b93-475c-b6c7-ba8fd6240d8c None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:01.956385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 705/1386] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:01.961689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07471df-bb55-4249-af2c-6f4e43432240 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:01.962085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07471df-bb55-4249-af2c-6f4e43432240 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:01.962269 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07471df-bb55-4249-af2c-6f4e43432240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:02.028482 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-a07471df-bb55-4249-af2c-6f4e43432240 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:02.029708 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a07471df-bb55-4249-af2c-6f4e43432240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:02.134163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 682/1387] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:02.436868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd19090-6b93-475c-b6c7-ba8fd6240d8c None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:02.438966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5dd19090-6b93-475c-b6c7-ba8fd6240d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:02.515253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:02.583908 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd19090-6b93-475c-b6c7-ba8fd6240d8c None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:02.700809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdfc6f-1518-4782-82eb-08206d70122a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:02.701309 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdfc6f-1518-4782-82eb-08206d70122a service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:02.701584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdfc6f-1518-4782-82eb-08206d70122a service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:02.701850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdfc6f-1518-4782-82eb-08206d70122a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:02.703462 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6bcdfc6f-1518-4782-82eb-08206d70122a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:02.742761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:02.793700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcdfc6f-1518-4782-82eb-08206d70122a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:02.796187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcdfc6f-1518-4782-82eb-08206d70122a service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:02.815273 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 706/1388] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:03.207563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3151b-c018-4a44-940d-65a6527cf089 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:03.208137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3151b-c018-4a44-940d-65a6527cf089 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:03.208137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3151b-c018-4a44-940d-65a6527cf089 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:03.209764 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 683/1389] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:03.215328 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f59372-cc1a-4f14-a7ac-317b8e709712 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:03.215942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f59372-cc1a-4f14-a7ac-317b8e709712 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:03.215942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f59372-cc1a-4f14-a7ac-317b8e709712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:03.303037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-a9f59372-cc1a-4f14-a7ac-317b8e709712 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:03.304160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a9f59372-cc1a-4f14-a7ac-317b8e709712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:03.409181 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 707/1390] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:03.713180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776df83-0577-46aa-bc43-4bf3b726dfec None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:03.713471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776df83-0577-46aa-bc43-4bf3b726dfec None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:03.713747 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776df83-0577-46aa-bc43-4bf3b726dfec None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:03.715142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 684/1391] 10.222.0.46 () {54 vars in 966 bytes} [Fri Sep 9 12:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:03.720483 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83623c02-4075-499e-a440-53f8229d50d8 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:03.720772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83623c02-4075-499e-a440-53f8229d50d8 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:03.721048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83623c02-4075-499e-a440-53f8229d50d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:03.737334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:03.796309 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-83623c02-4075-499e-a440-53f8229d50d8 None None] MFA Rules not processed for user `eff8fcb7fa2740bbb801badf800d2d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:03.797535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-83623c02-4075-499e-a440-53f8229d50d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:03.874784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 708/1392] 10.222.0.46 () {58 vars in 1063 bytes} [Fri Sep 9 12:49:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:03.884418 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776df83-0577-46aa-bc43-4bf3b726dfec None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:03.886020 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3776df83-0577-46aa-bc43-4bf3b726dfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:03.929335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:04.005052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776df83-0577-46aa-bc43-4bf3b726dfec None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:04.080380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba502c-e037-44a9-a506-e7b6af4e12d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff8fcb7fa2740bbb801badf800d2d6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:04.080984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba502c-e037-44a9-a506-e7b6af4e12d0 service glance] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:04.081267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba502c-e037-44a9-a506-e7b6af4e12d0 service glance] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:04.081544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba502c-e037-44a9-a506-e7b6af4e12d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:04.083167 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-10ba502c-e037-44a9-a506-e7b6af4e12d0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:04.167204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:04.199974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba502c-e037-44a9-a506-e7b6af4e12d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:04.201438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba502c-e037-44a9-a506-e7b6af4e12d0 service glance] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:04.220745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 685/1393] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:05.137351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2c348-ea59-4f54-a34a-36358ee49747 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:05.137857 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2c348-ea59-4f54-a34a-36358ee49747 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:05.137966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2c348-ea59-4f54-a34a-36358ee49747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:05.206145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-86f2c348-ea59-4f54-a34a-36358ee49747 None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:05.207358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-86f2c348-ea59-4f54-a34a-36358ee49747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:05.211057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc1c9-4e41-4709-b030-15550c97e8d7 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:05.211469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc1c9-4e41-4709-b030-15550c97e8d7 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:05.211747 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc1c9-4e41-4709-b030-15550c97e8d7 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:05.213102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 686/1394] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:05.218027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3e6a3-91b0-433b-890e-b5f7b3300961 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:05.218316 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3e6a3-91b0-433b-890e-b5f7b3300961 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:05.218627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3e6a3-91b0-433b-890e-b5f7b3300961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:05.265613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:05.288137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-20a3e6a3-91b0-433b-890e-b5f7b3300961 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:05.289337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-20a3e6a3-91b0-433b-890e-b5f7b3300961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:05.300482 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 709/1395] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 12:49:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:05.359355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3822-2de3-439e-8fb7-6fcdf1b8f761 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:05.359638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3822-2de3-439e-8fb7-6fcdf1b8f761 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:05.359909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3822-2de3-439e-8fb7-6fcdf1b8f761 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:05.361300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 710/1396] 10.222.0.46 () {54 vars in 943 bytes} [Fri Sep 9 12:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:49:05.368072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bda78-ef70-4ded-a19c-f88ee8e91973 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:05.368467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bda78-ef70-4ded-a19c-f88ee8e91973 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:05.368807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bda78-ef70-4ded-a19c-f88ee8e91973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:05.395808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 687/1397] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:49:05.448972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-7e8bda78-ef70-4ded-a19c-f88ee8e91973 None None] MFA Rules not processed for user `e8e4805f0b1e484c98fb399313f88ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:05.450160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7e8bda78-ef70-4ded-a19c-f88ee8e91973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:05.556588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 711/1398] 10.222.0.46 () {58 vars in 1040 bytes} [Fri Sep 9 12:49:05 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:49:05.565823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3e6a3-91b0-433b-890e-b5f7b3300961 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:05.567119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-20a3e6a3-91b0-433b-890e-b5f7b3300961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:05.606941 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:05.669669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3e6a3-91b0-433b-890e-b5f7b3300961 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:05.690449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8bda78-ef70-4ded-a19c-f88ee8e91973 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:05.691823 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7e8bda78-ef70-4ded-a19c-f88ee8e91973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:05.724554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:05.776814 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8e7cf-37a3-4d32-a694-8a80dc4e7cba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:05.777452 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8e7cf-37a3-4d32-a694-8a80dc4e7cba service swift] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:05.777728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8e7cf-37a3-4d32-a694-8a80dc4e7cba service swift] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:05.777994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8e7cf-37a3-4d32-a694-8a80dc4e7cba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:05.779640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c4c8e7cf-37a3-4d32-a694-8a80dc4e7cba service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:05.791866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8bda78-ef70-4ded-a19c-f88ee8e91973 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:05.835030 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:05.863215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8e7cf-37a3-4d32-a694-8a80dc4e7cba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:05.865415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8e7cf-37a3-4d32-a694-8a80dc4e7cba service swift] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:05.873555 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 688/1399] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 12:49:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:49:05.910008 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c07d0-2b57-4d08-9e55-81e19c6660aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:05.913893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c07d0-2b57-4d08-9e55-81e19c6660aa service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:05.913893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c07d0-2b57-4d08-9e55-81e19c6660aa service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:05.913893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c07d0-2b57-4d08-9e55-81e19c6660aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:05.913893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-379c07d0-2b57-4d08-9e55-81e19c6660aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:05.959841 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:06.007052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c07d0-2b57-4d08-9e55-81e19c6660aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:06.008624 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c07d0-2b57-4d08-9e55-81e19c6660aa service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:06.026869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 712/1400] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:49:08.362238 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f7c2a-626a-4176-91cb-9f91c2e12a8c None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:08.364506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f7c2a-626a-4176-91cb-9f91c2e12a8c None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:08.364788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f7c2a-626a-4176-91cb-9f91c2e12a8c None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:08.366166 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 689/1401] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:49:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:08.371706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d00abf-f419-4c1f-9bd2-4821d792828d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:08.372167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d00abf-f419-4c1f-9bd2-4821d792828d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:08.372254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d00abf-f419-4c1f-9bd2-4821d792828d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:08.445500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-e8d00abf-f419-4c1f-9bd2-4821d792828d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:08.446832 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e8d00abf-f419-4c1f-9bd2-4821d792828d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:08.558995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 713/1402] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:08.883361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f7c2a-626a-4176-91cb-9f91c2e12a8c None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:08.884874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-775f7c2a-626a-4176-91cb-9f91c2e12a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:09.041430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f7c2a-626a-4176-91cb-9f91c2e12a8c None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:09.096711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:09.184075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a785f-c949-4d0b-afed-08b258c36f06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:09.184697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a785f-c949-4d0b-afed-08b258c36f06 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:09.184980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a785f-c949-4d0b-afed-08b258c36f06 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:09.185250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a785f-c949-4d0b-afed-08b258c36f06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:09.186899 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9a1a785f-c949-4d0b-afed-08b258c36f06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:09.289516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a785f-c949-4d0b-afed-08b258c36f06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:09.291168 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a785f-c949-4d0b-afed-08b258c36f06 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:09.310257 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 690/1403] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:11.715093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8f7b2-fb1b-4a48-80f9-9fad134fc1d6 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:11.715668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8f7b2-fb1b-4a48-80f9-9fad134fc1d6 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:11.715668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8f7b2-fb1b-4a48-80f9-9fad134fc1d6 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:11.717012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 714/1404] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:11.727272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e75795-baa0-465d-a73b-d00ed364ecce None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:11.727782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e75795-baa0-465d-a73b-d00ed364ecce None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:11.727876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e75795-baa0-465d-a73b-d00ed364ecce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:11.799152 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-23e75795-baa0-465d-a73b-d00ed364ecce None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:11.802962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-23e75795-baa0-465d-a73b-d00ed364ecce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:11.803653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:11.912442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 691/1405] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:49:12.208761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8f7b2-fb1b-4a48-80f9-9fad134fc1d6 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:12.210314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7fe8f7b2-fb1b-4a48-80f9-9fad134fc1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:12.236464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:12.380496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8f7b2-fb1b-4a48-80f9-9fad134fc1d6 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:12.482179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:12.507380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19833201-9c7b-4476-97d0-340882fbcc48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:12.508079 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19833201-9c7b-4476-97d0-340882fbcc48 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:12.508423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19833201-9c7b-4476-97d0-340882fbcc48 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:12.508767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19833201-9c7b-4476-97d0-340882fbcc48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:12.510674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-19833201-9c7b-4476-97d0-340882fbcc48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:12.620249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19833201-9c7b-4476-97d0-340882fbcc48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:12.621989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19833201-9c7b-4476-97d0-340882fbcc48 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:12.641173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 715/1406] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:15.072631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f1b09-820f-4f3e-b00d-23a76ddb614a None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:15.073195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f1b09-820f-4f3e-b00d-23a76ddb614a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:15.074896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f1b09-820f-4f3e-b00d-23a76ddb614a None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:15.075395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 692/1407] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:49:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:15.080983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8da409-05f9-4666-b3b1-4270774b5c48 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:15.081837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8da409-05f9-4666-b3b1-4270774b5c48 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:15.082310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8da409-05f9-4666-b3b1-4270774b5c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:15.164957 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-5c8da409-05f9-4666-b3b1-4270774b5c48 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:15.166516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5c8da409-05f9-4666-b3b1-4270774b5c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:15.222026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:15.271145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 716/1408] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:15.561268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f1b09-820f-4f3e-b00d-23a76ddb614a None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:15.562881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6f4f1b09-820f-4f3e-b00d-23a76ddb614a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:15.716534 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:15.729829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f1b09-820f-4f3e-b00d-23a76ddb614a None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:15.851039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc49c2-cf51-4daf-b589-b819bb9d5b8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:15.851557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc49c2-cf51-4daf-b589-b819bb9d5b8f service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:15.851841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc49c2-cf51-4daf-b589-b819bb9d5b8f service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:15.852115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc49c2-cf51-4daf-b589-b819bb9d5b8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:15.853709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-77bc49c2-cf51-4daf-b589-b819bb9d5b8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:15.953331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc49c2-cf51-4daf-b589-b819bb9d5b8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:15.955806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc49c2-cf51-4daf-b589-b819bb9d5b8f service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:15.967437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:15.978007 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 693/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:25.299749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=19 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:26.030049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:29.784691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04b8c2-1392-431c-a1d3-aa7690547073 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:29.785225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04b8c2-1392-431c-a1d3-aa7690547073 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:29.785225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04b8c2-1392-431c-a1d3-aa7690547073 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:29.786869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 717/1410] 10.222.0.46 () {54 vars in 918 bytes} [Fri Sep 9 12:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:29.791661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbd879-56d1-4f39-93ed-9a04696dc625 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:29.795094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbd879-56d1-4f39-93ed-9a04696dc625 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:29.795094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbd879-56d1-4f39-93ed-9a04696dc625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:29.868001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-d2fbd879-56d1-4f39-93ed-9a04696dc625 None None] MFA Rules not processed for user `33e177d86450489cacc86578ba26509d`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:29.869221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d2fbd879-56d1-4f39-93ed-9a04696dc625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:29.976121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 694/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Fri Sep 9 12:49:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:29.990550 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c10ee-c2d5-4c10-b61a-b158c3514368 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:29.990836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c10ee-c2d5-4c10-b61a-b158c3514368 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:29.991110 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c10ee-c2d5-4c10-b61a-b158c3514368 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:29.992430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 718/1412] 10.222.0.46 () {54 vars in 918 bytes} [Fri Sep 9 12:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:29.996866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270eb37-abf4-4c83-8ba4-bea13e4d8dbd None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:29.997151 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270eb37-abf4-4c83-8ba4-bea13e4d8dbd None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:29.997421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270eb37-abf4-4c83-8ba4-bea13e4d8dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:30.070707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:30.073980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-6270eb37-abf4-4c83-8ba4-bea13e4d8dbd None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:30.075205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6270eb37-abf4-4c83-8ba4-bea13e4d8dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:30.194147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 695/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Fri Sep 9 12:49:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:32.802744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b563b-fe07-4018-9386-eda0a6401df9 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:32.803313 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b563b-fe07-4018-9386-eda0a6401df9 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:32.803313 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b563b-fe07-4018-9386-eda0a6401df9 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:32.804615 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:32.809831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fda61-4d30-4063-88d8-01bf79a2b845 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:32.810278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fda61-4d30-4063-88d8-01bf79a2b845 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:32.810375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fda61-4d30-4063-88d8-01bf79a2b845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:32.877266 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-1e6fda61-4d30-4063-88d8-01bf79a2b845 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:32.878464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-1e6fda61-4d30-4063-88d8-01bf79a2b845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:32.932576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:32.974922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:33.269889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b563b-fe07-4018-9386-eda0a6401df9 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:33.271435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b45b563b-fe07-4018-9386-eda0a6401df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:33.420074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b563b-fe07-4018-9386-eda0a6401df9 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:33.492502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:33.533107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951070d5-ef40-489e-ae98-e6f81541a421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:33.533706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951070d5-ef40-489e-ae98-e6f81541a421 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:33.533987 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951070d5-ef40-489e-ae98-e6f81541a421 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:33.534260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951070d5-ef40-489e-ae98-e6f81541a421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:33.535955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-951070d5-ef40-489e-ae98-e6f81541a421 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:33.624656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951070d5-ef40-489e-ae98-e6f81541a421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:33.627317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951070d5-ef40-489e-ae98-e6f81541a421 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:33.645890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 720/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:34.784604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94839c2-e0ef-4d2f-a0e9-bc1849e083d2 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:34.785107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94839c2-e0ef-4d2f-a0e9-bc1849e083d2 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:34.785203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94839c2-e0ef-4d2f-a0e9-bc1849e083d2 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:34.786533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:34.792593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfdbe5-c0bf-4adc-8365-209369f46bb1 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:34.793050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfdbe5-c0bf-4adc-8365-209369f46bb1 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:34.793164 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfdbe5-c0bf-4adc-8365-209369f46bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:34.868145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0ebfdbe5-c0bf-4adc-8365-209369f46bb1 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:34.869343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0ebfdbe5-c0bf-4adc-8365-209369f46bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:34.895503 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:34.964388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:35.122075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94839c2-e0ef-4d2f-a0e9-bc1849e083d2 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:35.123521 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c94839c2-e0ef-4d2f-a0e9-bc1849e083d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:35.269466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94839c2-e0ef-4d2f-a0e9-bc1849e083d2 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:35.309379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:35.382983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef61c6e-4e2a-40af-b9ad-a1b45ae73e67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:35.383508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61c6e-4e2a-40af-b9ad-a1b45ae73e67 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:35.383790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61c6e-4e2a-40af-b9ad-a1b45ae73e67 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:35.384059 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61c6e-4e2a-40af-b9ad-a1b45ae73e67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:35.385658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-8ef61c6e-4e2a-40af-b9ad-a1b45ae73e67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:35.478845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef61c6e-4e2a-40af-b9ad-a1b45ae73e67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:35.481026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef61c6e-4e2a-40af-b9ad-a1b45ae73e67 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:35.498242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 698/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:49:36.636376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb13150-9199-4073-bab1-4980515e47bc None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:36.636947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb13150-9199-4073-bab1-4980515e47bc None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:36.636947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb13150-9199-4073-bab1-4980515e47bc None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:36.638321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:36.644947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b409d-3b57-4c0a-8c3b-d18af1732d19 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:36.645420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b409d-3b57-4c0a-8c3b-d18af1732d19 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:36.645534 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b409d-3b57-4c0a-8c3b-d18af1732d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:36.701268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:36.723174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-eb6b409d-3b57-4c0a-8c3b-d18af1732d19 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:36.724390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-eb6b409d-3b57-4c0a-8c3b-d18af1732d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:36.820137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:36.971656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb13150-9199-4073-bab1-4980515e47bc None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:36.973156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ecb13150-9199-4073-bab1-4980515e47bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:37.117417 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb13150-9199-4073-bab1-4980515e47bc None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:37.129685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:37.229011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300b897-35b6-4f8f-8216-4aa11cf5e09c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:37.229533 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300b897-35b6-4f8f-8216-4aa11cf5e09c service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:37.229811 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300b897-35b6-4f8f-8216-4aa11cf5e09c service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:37.230080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300b897-35b6-4f8f-8216-4aa11cf5e09c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:37.231703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8300b897-35b6-4f8f-8216-4aa11cf5e09c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:37.320472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300b897-35b6-4f8f-8216-4aa11cf5e09c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:37.322053 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300b897-35b6-4f8f-8216-4aa11cf5e09c service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:37.340660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 723/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:38.468632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a937172-229d-4140-8d00-e788265a9370 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:38.469190 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a937172-229d-4140-8d00-e788265a9370 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:38.469190 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a937172-229d-4140-8d00-e788265a9370 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:38.471146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:49:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:38.477178 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203e627-29dd-49a6-9cbc-cd24757e86d4 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:38.477583 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203e627-29dd-49a6-9cbc-cd24757e86d4 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:38.477715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203e627-29dd-49a6-9cbc-cd24757e86d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:38.497030 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:38.552855 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-4203e627-29dd-49a6-9cbc-cd24757e86d4 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:38.553969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4203e627-29dd-49a6-9cbc-cd24757e86d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:38.650767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:38.798291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a937172-229d-4140-8d00-e788265a9370 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:38.799873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-1a937172-229d-4140-8d00-e788265a9370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:38.910827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:38.946088 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a937172-229d-4140-8d00-e788265a9370 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:39.058112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3549a7b-db52-431a-a13b-c1de57619c67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:39.058754 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3549a7b-db52-431a-a13b-c1de57619c67 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:39.059038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3549a7b-db52-431a-a13b-c1de57619c67 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:39.059315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3549a7b-db52-431a-a13b-c1de57619c67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:39.060915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e3549a7b-db52-431a-a13b-c1de57619c67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:39.137787 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:39.150812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3549a7b-db52-431a-a13b-c1de57619c67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:39.152409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3549a7b-db52-431a-a13b-c1de57619c67 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:39.170354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 701/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:40.316077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fff428-b093-434f-bc40-6d58f9322234 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:40.316703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fff428-b093-434f-bc40-6d58f9322234 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:40.316703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fff428-b093-434f-bc40-6d58f9322234 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:40.317948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:40.324458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3383e-8d6d-44a8-bbfc-aed39f099b29 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:40.324926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3383e-8d6d-44a8-bbfc-aed39f099b29 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:40.325036 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3383e-8d6d-44a8-bbfc-aed39f099b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:40.399891 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-77a3383e-8d6d-44a8-bbfc-aed39f099b29 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:40.401001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-77a3383e-8d6d-44a8-bbfc-aed39f099b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:40.496665 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:40.647211 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fff428-b093-434f-bc40-6d58f9322234 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:40.648721 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-50fff428-b093-434f-bc40-6d58f9322234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:40.732104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:40.793827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fff428-b093-434f-bc40-6d58f9322234 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:40.906386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0355efd-43de-413b-820d-e941eb0d3bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:40.907628 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0355efd-43de-413b-820d-e941eb0d3bd2 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:40.908064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0355efd-43de-413b-820d-e941eb0d3bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:40.908340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0355efd-43de-413b-820d-e941eb0d3bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:40.909938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a0355efd-43de-413b-820d-e941eb0d3bd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:40.957931 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:40.998060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0355efd-43de-413b-820d-e941eb0d3bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:40.999593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0355efd-43de-413b-820d-e941eb0d3bd2 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:41.017488 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 726/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:42.118386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bfa29-8205-4efb-aa0f-87d80280dac8 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:42.118967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bfa29-8205-4efb-aa0f-87d80280dac8 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:42.118967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bfa29-8205-4efb-aa0f-87d80280dac8 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:42.120302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:42.127632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f15fe8-e0f3-40f6-8bc8-0a69b4e7e2f0 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:42.128083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f15fe8-e0f3-40f6-8bc8-0a69b4e7e2f0 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:42.128208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f15fe8-e0f3-40f6-8bc8-0a69b4e7e2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:42.206010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-83f15fe8-e0f3-40f6-8bc8-0a69b4e7e2f0 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:42.207285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-83f15fe8-e0f3-40f6-8bc8-0a69b4e7e2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:42.304586 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:42.455186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bfa29-8205-4efb-aa0f-87d80280dac8 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:42.456777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-bf6bfa29-8205-4efb-aa0f-87d80280dac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:42.506398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:42.602005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bfa29-8205-4efb-aa0f-87d80280dac8 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:42.715409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115fc4a1-411b-4d21-8c1c-54b60d8d9a06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:42.716013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fc4a1-411b-4d21-8c1c-54b60d8d9a06 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:42.716291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fc4a1-411b-4d21-8c1c-54b60d8d9a06 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:42.716562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fc4a1-411b-4d21-8c1c-54b60d8d9a06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:42.718166 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-115fc4a1-411b-4d21-8c1c-54b60d8d9a06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:42.725411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:42.809456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115fc4a1-411b-4d21-8c1c-54b60d8d9a06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:42.811009 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115fc4a1-411b-4d21-8c1c-54b60d8d9a06 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:42.829763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 704/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:43.965905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9d048-b891-4f7d-8f37-101807eb28b0 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:43.966418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9d048-b891-4f7d-8f37-101807eb28b0 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:43.966567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9d048-b891-4f7d-8f37-101807eb28b0 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:43.967892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:43.974363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d686cb2-33c9-45b9-9da3-faccbe2e8a46 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:43.974816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d686cb2-33c9-45b9-9da3-faccbe2e8a46 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:43.974926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d686cb2-33c9-45b9-9da3-faccbe2e8a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:44.050722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-4d686cb2-33c9-45b9-9da3-faccbe2e8a46 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:44.051831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-4d686cb2-33c9-45b9-9da3-faccbe2e8a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:44.119961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:44.150634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:44.302977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9d048-b891-4f7d-8f37-101807eb28b0 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:44.304494 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-3ad9d048-b891-4f7d-8f37-101807eb28b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:44.321927 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:44.447930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9d048-b891-4f7d-8f37-101807eb28b0 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:44.533702 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:44.561157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e627afb-e162-431d-953c-1f56cc682ce0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:44.561755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e627afb-e162-431d-953c-1f56cc682ce0 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:44.562035 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e627afb-e162-431d-953c-1f56cc682ce0 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:44.562330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e627afb-e162-431d-953c-1f56cc682ce0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:44.564711 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0e627afb-e162-431d-953c-1f56cc682ce0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:44.653364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e627afb-e162-431d-953c-1f56cc682ce0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:44.655002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e627afb-e162-431d-953c-1f56cc682ce0 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:44.673623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 729/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:45.788333 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03c81c-5c85-4e4f-b74c-5976c5ba1672 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:45.788911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03c81c-5c85-4e4f-b74c-5976c5ba1672 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:45.788911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03c81c-5c85-4e4f-b74c-5976c5ba1672 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:45.790205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:45.796340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee4528-d985-4cb5-8af0-e0f6d8e28844 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:45.796760 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee4528-d985-4cb5-8af0-e0f6d8e28844 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:45.796942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee4528-d985-4cb5-8af0-e0f6d8e28844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:45.872189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-f4ee4528-d985-4cb5-8af0-e0f6d8e28844 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:45.873382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f4ee4528-d985-4cb5-8af0-e0f6d8e28844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:45.919362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:45.973409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:49:46.122764 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03c81c-5c85-4e4f-b74c-5976c5ba1672 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:46.124202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-df03c81c-5c85-4e4f-b74c-5976c5ba1672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:46.280231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03c81c-5c85-4e4f-b74c-5976c5ba1672 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:46.334697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:46.391906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e9ed96-5285-49b4-8bb4-e6531b158d35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:46.392432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9ed96-5285-49b4-8bb4-e6531b158d35 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:46.392734 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9ed96-5285-49b4-8bb4-e6531b158d35 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:46.393002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9ed96-5285-49b4-8bb4-e6531b158d35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:46.394611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-95e9ed96-5285-49b4-8bb4-e6531b158d35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:46.483561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e9ed96-5285-49b4-8bb4-e6531b158d35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:46.485819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e9ed96-5285-49b4-8bb4-e6531b158d35 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:46.503720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 707/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:47.626453 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85700953-9c8e-4406-8718-15ee52116e1a None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:47.627108 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85700953-9c8e-4406-8718-15ee52116e1a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:47.627108 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85700953-9c8e-4406-8718-15ee52116e1a None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:47.628336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:49:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:47.634456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040237f-d8cf-49e6-9410-28798c7417b2 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:47.634902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040237f-d8cf-49e6-9410-28798c7417b2 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:47.635023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040237f-d8cf-49e6-9410-28798c7417b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:47.710256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-9040237f-d8cf-49e6-9410-28798c7417b2 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:47.711471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9040237f-d8cf-49e6-9410-28798c7417b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:47.713470 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:47.813598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:47.964081 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85700953-9c8e-4406-8718-15ee52116e1a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:47.965591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-85700953-9c8e-4406-8718-15ee52116e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:48.112570 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85700953-9c8e-4406-8718-15ee52116e1a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:48.146115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:48.235691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bea07e-3996-4dd9-89d7-457959bba689 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:48.236302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bea07e-3996-4dd9-89d7-457959bba689 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:48.236580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bea07e-3996-4dd9-89d7-457959bba689 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:48.236851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bea07e-3996-4dd9-89d7-457959bba689 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:48.238457 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-06bea07e-3996-4dd9-89d7-457959bba689 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:48.327296 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bea07e-3996-4dd9-89d7-457959bba689 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:48.329408 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bea07e-3996-4dd9-89d7-457959bba689 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:48.347885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 732/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:50.432723 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce958d01-aad2-4760-8518-ec0e742fa93e None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:50.433300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce958d01-aad2-4760-8518-ec0e742fa93e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:50.433300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce958d01-aad2-4760-8518-ec0e742fa93e None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:50.434623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:50.440695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a043a47-ce6b-448b-8e16-f003e1e08d07 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:50.441192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a043a47-ce6b-448b-8e16-f003e1e08d07 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:50.441296 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a043a47-ce6b-448b-8e16-f003e1e08d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:50.481501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:50.507034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-8a043a47-ce6b-448b-8e16-f003e1e08d07 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:50.508243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8a043a47-ce6b-448b-8e16-f003e1e08d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:50.610228 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:50.908357 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce958d01-aad2-4760-8518-ec0e742fa93e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:50.909884 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ce958d01-aad2-4760-8518-ec0e742fa93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:51.013502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce958d01-aad2-4760-8518-ec0e742fa93e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:51.045143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:51.086073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687cf0c-50bb-4e44-99ae-c05168badbce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:51.086698 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687cf0c-50bb-4e44-99ae-c05168badbce service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:51.086983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687cf0c-50bb-4e44-99ae-c05168badbce service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:51.087260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687cf0c-50bb-4e44-99ae-c05168badbce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:51.088847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-1687cf0c-50bb-4e44-99ae-c05168badbce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:51.199850 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687cf0c-50bb-4e44-99ae-c05168badbce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:51.201229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687cf0c-50bb-4e44-99ae-c05168badbce service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:51.220142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 710/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:53.502285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb2ba3-3549-4465-a986-023c4ff0b0f8 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:53.502880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb2ba3-3549-4465-a986-023c4ff0b0f8 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:53.502976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb2ba3-3549-4465-a986-023c4ff0b0f8 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:53.504290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:53.509768 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c53a0-9856-4c66-8d40-7ba911cf05a9 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:53.510172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c53a0-9856-4c66-8d40-7ba911cf05a9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:53.510351 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c53a0-9856-4c66-8d40-7ba911cf05a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:53.576774 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-d26c53a0-9856-4c66-8d40-7ba911cf05a9 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:53.577972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d26c53a0-9856-4c66-8d40-7ba911cf05a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:53.591017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:53.674450 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:53.965835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb2ba3-3549-4465-a986-023c4ff0b0f8 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:53.967391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-08eb2ba3-3549-4465-a986-023c4ff0b0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:54.068215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb2ba3-3549-4465-a986-023c4ff0b0f8 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:54.081010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:54.143265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774226d7-e01d-4cec-b998-cda4c42ecea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:54.143869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774226d7-e01d-4cec-b998-cda4c42ecea1 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:54.144149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774226d7-e01d-4cec-b998-cda4c42ecea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:54.144422 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774226d7-e01d-4cec-b998-cda4c42ecea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:54.146016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-774226d7-e01d-4cec-b998-cda4c42ecea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:54.257466 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774226d7-e01d-4cec-b998-cda4c42ecea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:54.258938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774226d7-e01d-4cec-b998-cda4c42ecea1 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:54.279464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 735/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:49:56.819232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427096f-2624-47e7-831a-7fb61b06ce18 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:56.819806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427096f-2624-47e7-831a-7fb61b06ce18 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:56.819806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427096f-2624-47e7-831a-7fb61b06ce18 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:56.821101 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:49:56.827051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac1785-54f5-4883-93c0-533b7f8acffd None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:56.827492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac1785-54f5-4883-93c0-533b7f8acffd None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:56.827595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac1785-54f5-4883-93c0-533b7f8acffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:56.874803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:56.893223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-a1ac1785-54f5-4883-93c0-533b7f8acffd None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:56.894421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a1ac1785-54f5-4883-93c0-533b7f8acffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:56.991653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:49:57.278958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427096f-2624-47e7-831a-7fb61b06ce18 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:49:57.280554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e427096f-2624-47e7-831a-7fb61b06ce18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:57.385453 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427096f-2624-47e7-831a-7fb61b06ce18 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:49:57.427905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:49:57.462672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46159cc-0735-4736-960b-ce9641697ced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:49:57.463308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46159cc-0735-4736-960b-ce9641697ced service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:57.463587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46159cc-0735-4736-960b-ce9641697ced service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:57.463884 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46159cc-0735-4736-960b-ce9641697ced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:57.465483 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a46159cc-0735-4736-960b-ce9641697ced service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:57.575524 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46159cc-0735-4736-960b-ce9641697ced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:49:57.576959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46159cc-0735-4736-960b-ce9641697ced service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:49:57.596260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 713/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:49:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:49:59.884173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b5844-f88d-4ab7-a255-ebccd9478c00 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:59.884734 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b5844-f88d-4ab7-a255-ebccd9478c00 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:59.884734 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b5844-f88d-4ab7-a255-ebccd9478c00 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:59.886071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:49:59.891432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd527063-fab6-45cc-92c3-c898c2bbc818 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:49:59.891831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd527063-fab6-45cc-92c3-c898c2bbc818 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:49:59.892014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd527063-fab6-45cc-92c3-c898c2bbc818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:49:59.958808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-fd527063-fab6-45cc-92c3-c898c2bbc818 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:49:59.959915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-fd527063-fab6-45cc-92c3-c898c2bbc818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:49:59.979803 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:00.055690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:00.341906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b5844-f88d-4ab7-a255-ebccd9478c00 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:00.343770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-753b5844-f88d-4ab7-a255-ebccd9478c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:00.447134 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b5844-f88d-4ab7-a255-ebccd9478c00 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:00.468064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:00.520983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7330233-a227-4c39-a8a1-7e8b89dd05b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:00.521586 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7330233-a227-4c39-a8a1-7e8b89dd05b1 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:00.521858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7330233-a227-4c39-a8a1-7e8b89dd05b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:00.522138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7330233-a227-4c39-a8a1-7e8b89dd05b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:00.523773 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a7330233-a227-4c39-a8a1-7e8b89dd05b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:00.631293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7330233-a227-4c39-a8a1-7e8b89dd05b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:00.632606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7330233-a227-4c39-a8a1-7e8b89dd05b1 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:00.650870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 738/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:50:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:50:02.923862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7139a8-7804-4e93-90b5-50cb35a71fe0 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:02.924499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7139a8-7804-4e93-90b5-50cb35a71fe0 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:02.924499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7139a8-7804-4e93-90b5-50cb35a71fe0 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:02.925731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:50:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:02.931412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb715e70-94b8-44da-928b-4306230eb7ad None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:02.931828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb715e70-94b8-44da-928b-4306230eb7ad None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:02.931982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb715e70-94b8-44da-928b-4306230eb7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:02.994698 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:02.997273 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-bb715e70-94b8-44da-928b-4306230eb7ad None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:02.998512 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-bb715e70-94b8-44da-928b-4306230eb7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:03.095826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:03.391325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7139a8-7804-4e93-90b5-50cb35a71fe0 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:03.392927 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-4d7139a8-7804-4e93-90b5-50cb35a71fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:03.493821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7139a8-7804-4e93-90b5-50cb35a71fe0 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:03.542273 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:03.566267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c39de2-f765-408a-aa66-5b1ad407ee70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:03.566897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c39de2-f765-408a-aa66-5b1ad407ee70 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:03.567173 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c39de2-f765-408a-aa66-5b1ad407ee70 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:03.567441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c39de2-f765-408a-aa66-5b1ad407ee70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:03.569006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f6c39de2-f765-408a-aa66-5b1ad407ee70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:03.680292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c39de2-f765-408a-aa66-5b1ad407ee70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:03.681639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c39de2-f765-408a-aa66-5b1ad407ee70 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:03.699965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 716/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:50:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:50:05.912973 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1904aeb-1672-4596-b006-61ee7668231b None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:05.913545 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1904aeb-1672-4596-b006-61ee7668231b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:05.913545 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1904aeb-1672-4596-b006-61ee7668231b None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:05.914887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:05.920122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be9480-37f9-4ad3-ab30-1e7bd2b0bfec None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:05.920510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be9480-37f9-4ad3-ab30-1e7bd2b0bfec None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:05.920700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be9480-37f9-4ad3-ab30-1e7bd2b0bfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:05.990776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-a5be9480-37f9-4ad3-ab30-1e7bd2b0bfec None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:05.991970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a5be9480-37f9-4ad3-ab30-1e7bd2b0bfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:06.020472 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:06.086468 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:06.375785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1904aeb-1672-4596-b006-61ee7668231b None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:06.377341 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d1904aeb-1672-4596-b006-61ee7668231b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:06.480579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1904aeb-1672-4596-b006-61ee7668231b None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:06.509070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:06.553661 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f6ac0-98b2-489f-8a72-19646b957d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:06.554263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f6ac0-98b2-489f-8a72-19646b957d11 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:06.554571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f6ac0-98b2-489f-8a72-19646b957d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:06.554843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f6ac0-98b2-489f-8a72-19646b957d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:06.556433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-249f6ac0-98b2-489f-8a72-19646b957d11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:06.665079 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f6ac0-98b2-489f-8a72-19646b957d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:06.666429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f6ac0-98b2-489f-8a72-19646b957d11 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:06.685037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 741/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:50:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:50:08.901954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc4258-0f54-4072-aae5-142c5ac19c99 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:08.902557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc4258-0f54-4072-aae5-142c5ac19c99 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:08.902557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc4258-0f54-4072-aae5-142c5ac19c99 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:08.903879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:08.909595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0c5f8-ec7f-4ba8-989a-0d08803d3d9f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:08.910030 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0c5f8-ec7f-4ba8-989a-0d08803d3d9f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:08.910137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0c5f8-ec7f-4ba8-989a-0d08803d3d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:08.984081 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-92b0c5f8-ec7f-4ba8-989a-0d08803d3d9f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:08.985868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-92b0c5f8-ec7f-4ba8-989a-0d08803d3d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:08.994049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:09.082943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:09.366658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc4258-0f54-4072-aae5-142c5ac19c99 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:09.368229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-45fc4258-0f54-4072-aae5-142c5ac19c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:09.591448 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc4258-0f54-4072-aae5-142c5ac19c99 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:09.645575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:09.662387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72339371-c330-4786-adde-5079684d3819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:09.663019 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72339371-c330-4786-adde-5079684d3819 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:09.663293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72339371-c330-4786-adde-5079684d3819 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:09.663559 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72339371-c330-4786-adde-5079684d3819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:09.665136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-72339371-c330-4786-adde-5079684d3819 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:09.775644 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72339371-c330-4786-adde-5079684d3819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:09.777056 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72339371-c330-4786-adde-5079684d3819 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:09.794459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 719/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:50:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:50:12.006957 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46292197-e804-4862-bee5-d10c5641183d None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:12.007481 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46292197-e804-4862-bee5-d10c5641183d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:12.007481 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46292197-e804-4862-bee5-d10c5641183d None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:12.008846 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:12.014092 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681d8da-110e-4d14-a073-d1dc8d59d451 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:12.014600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681d8da-110e-4d14-a073-d1dc8d59d451 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:12.014710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681d8da-110e-4d14-a073-d1dc8d59d451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:12.081809 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-3681d8da-110e-4d14-a073-d1dc8d59d451 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:12.083034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3681d8da-110e-4d14-a073-d1dc8d59d451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:12.123220 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:12.180706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:12.462920 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46292197-e804-4862-bee5-d10c5641183d None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:12.464482 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-46292197-e804-4862-bee5-d10c5641183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:12.566176 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46292197-e804-4862-bee5-d10c5641183d None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:12.603625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:12.639088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4283b9f-080d-4061-b286-c9ba26fa49ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:12.639693 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4283b9f-080d-4061-b286-c9ba26fa49ea service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:12.639977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4283b9f-080d-4061-b286-c9ba26fa49ea service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:12.640252 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4283b9f-080d-4061-b286-c9ba26fa49ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:12.642388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d4283b9f-080d-4061-b286-c9ba26fa49ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:12.750596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4283b9f-080d-4061-b286-c9ba26fa49ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:12.752015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4283b9f-080d-4061-b286-c9ba26fa49ea service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:12.771554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 744/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:50:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:50:14.967757 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb5321-f88e-44b0-9f71-1bc2259c0c86 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:14.968358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb5321-f88e-44b0-9f71-1bc2259c0c86 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:14.968358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb5321-f88e-44b0-9f71-1bc2259c0c86 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:14.969804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:14.975333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdbb40-91c6-4449-b6b1-2b9fc720741e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:14.975788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdbb40-91c6-4449-b6b1-2b9fc720741e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:14.975896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdbb40-91c6-4449-b6b1-2b9fc720741e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:15.039833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-9dbdbb40-91c6-4449-b6b1-2b9fc720741e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:15.041019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9dbdbb40-91c6-4449-b6b1-2b9fc720741e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:15.058866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:15.136459 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:15.419196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb5321-f88e-44b0-9f71-1bc2259c0c86 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:15.420741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e5fb5321-f88e-44b0-9f71-1bc2259c0c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:15.523842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb5321-f88e-44b0-9f71-1bc2259c0c86 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:15.588253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:15.595779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e33569-c776-4f0b-b1a4-98a1fa29cd71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:15.596419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e33569-c776-4f0b-b1a4-98a1fa29cd71 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:15.596693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e33569-c776-4f0b-b1a4-98a1fa29cd71 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:15.596966 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e33569-c776-4f0b-b1a4-98a1fa29cd71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:15.598607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c2e33569-c776-4f0b-b1a4-98a1fa29cd71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:15.707251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e33569-c776-4f0b-b1a4-98a1fa29cd71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:15.708947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e33569-c776-4f0b-b1a4-98a1fa29cd71 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:15.726913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 722/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:50:17.937052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8527f-f77d-42f2-a7b2-2f6777a497db None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:17.937746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8527f-f77d-42f2-a7b2-2f6777a497db None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:17.937746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8527f-f77d-42f2-a7b2-2f6777a497db None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:17.939071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:17.944497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a763d0-096a-4f49-89c3-b3b28ea4ebfd None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:17.944896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a763d0-096a-4f49-89c3-b3b28ea4ebfd None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:17.945046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a763d0-096a-4f49-89c3-b3b28ea4ebfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:18.011814 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-51a763d0-096a-4f49-89c3-b3b28ea4ebfd None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:18.012995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-51a763d0-096a-4f49-89c3-b3b28ea4ebfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:18.060191 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:18.110828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:18.401352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8527f-f77d-42f2-a7b2-2f6777a497db None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:18.402971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c3f8527f-f77d-42f2-a7b2-2f6777a497db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:18.509094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8527f-f77d-42f2-a7b2-2f6777a497db None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:18.556878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:18.583153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3014e7a-2bf7-409f-ad56-35c9648f8e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:18.583752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3014e7a-2bf7-409f-ad56-35c9648f8e5e service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:18.584024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3014e7a-2bf7-409f-ad56-35c9648f8e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:18.584292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3014e7a-2bf7-409f-ad56-35c9648f8e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:18.585902 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c3014e7a-2bf7-409f-ad56-35c9648f8e5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:18.692464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3014e7a-2bf7-409f-ad56-35c9648f8e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:18.693876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3014e7a-2bf7-409f-ad56-35c9648f8e5e service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:18.712929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 747/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:50:28.109211 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=22 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:28.710566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=66 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:39.639563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b16946-480c-4415-8d92-cf7c99ba6312 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:39.640126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b16946-480c-4415-8d92-cf7c99ba6312 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:39.640126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b16946-480c-4415-8d92-cf7c99ba6312 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:39.642053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 724/1471] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:50:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:39.647745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7559c-36e4-4c5f-9db0-b55716bfbc42 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:39.648204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7559c-36e4-4c5f-9db0-b55716bfbc42 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:39.648304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7559c-36e4-4c5f-9db0-b55716bfbc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:39.837913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d6e7559c-36e4-4c5f-9db0-b55716bfbc42 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:39.839157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d6e7559c-36e4-4c5f-9db0-b55716bfbc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:39.935829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 748/1472] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:40.246215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44349b-9f0b-42de-901b-4311e4f9ce33 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:40.251467 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44349b-9f0b-42de-901b-4311e4f9ce33 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:40.251467 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44349b-9f0b-42de-901b-4311e4f9ce33 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:40.251467 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 725/1473] 10.222.0.46 () {54 vars in 966 bytes} [Fri Sep 9 12:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:40.253072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94160662-70f1-48e4-b13c-61d4e28ad18f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:40.253355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94160662-70f1-48e4-b13c-61d4e28ad18f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:40.253624 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94160662-70f1-48e4-b13c-61d4e28ad18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:40.332237 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-94160662-70f1-48e4-b13c-61d4e28ad18f None None] MFA Rules not processed for user `eff8fcb7fa2740bbb801badf800d2d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:40.333444 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-94160662-70f1-48e4-b13c-61d4e28ad18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:40.354612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:40.410031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 749/1474] 10.222.0.46 () {58 vars in 1063 bytes} [Fri Sep 9 12:50:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:40.417725 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44349b-9f0b-42de-901b-4311e4f9ce33 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:40.419317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-8b44349b-9f0b-42de-901b-4311e4f9ce33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:40.521453 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44349b-9f0b-42de-901b-4311e4f9ce33 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:40.601422 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a34ed-d961-4006-b0a6-b8fc072127e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff8fcb7fa2740bbb801badf800d2d6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:40.602021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a34ed-d961-4006-b0a6-b8fc072127e9 service glance] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:40.602299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a34ed-d961-4006-b0a6-b8fc072127e9 service glance] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:40.602600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a34ed-d961-4006-b0a6-b8fc072127e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:40.604188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-148a34ed-d961-4006-b0a6-b8fc072127e9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:40.656657 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:40.709729 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a34ed-d961-4006-b0a6-b8fc072127e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:40.711179 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a34ed-d961-4006-b0a6-b8fc072127e9 service glance] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:40.729380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 726/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:50:42.960438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da8940-5aff-46cc-9989-4b5c441d908a None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:42.961085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da8940-5aff-46cc-9989-4b5c441d908a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:42.961085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da8940-5aff-46cc-9989-4b5c441d908a None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:42.962299 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 750/1476] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:42.967513 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5b3bc-b1c1-4f3a-97bd-71ebc4fb8faf None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:42.967949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5b3bc-b1c1-4f3a-97bd-71ebc4fb8faf None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:42.968056 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5b3bc-b1c1-4f3a-97bd-71ebc4fb8faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:43.033770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-8fb5b3bc-b1c1-4f3a-97bd-71ebc4fb8faf None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:43.034998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-8fb5b3bc-b1c1-4f3a-97bd-71ebc4fb8faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:43.130981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 727/1477] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:43.405644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da8940-5aff-46cc-9989-4b5c441d908a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:43.407729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f1da8940-5aff-46cc-9989-4b5c441d908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:43.528388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da8940-5aff-46cc-9989-4b5c441d908a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:43.576093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:43.619447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdead480-2de6-4257-81fd-559bc8575780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:43.620035 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdead480-2de6-4257-81fd-559bc8575780 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:43.620303 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdead480-2de6-4257-81fd-559bc8575780 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:43.620566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdead480-2de6-4257-81fd-559bc8575780 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:43.621188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdead480-2de6-4257-81fd-559bc8575780 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:43.622919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: WARNING py.warnings [None req-fdead480-2de6-4257-81fd-559bc8575780 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:50:43.622919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: warnings.warn(msg) Sep 09 12:50:43.622919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: Sep 09 12:50:43.624026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdead480-2de6-4257-81fd-559bc8575780 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:43.643166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 751/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Fri Sep 9 12:50:43 2022] GET /identity/v3/projects => generated 2553 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:50:45.794750 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1c532-8467-45de-b0df-14ce74354719 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:45.795318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1c532-8467-45de-b0df-14ce74354719 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:45.795318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1c532-8467-45de-b0df-14ce74354719 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:45.796888 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 728/1479] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:45.802960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125b1bf-e7ae-43e3-bf1c-ebbc6c1eb2c3 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:45.803361 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125b1bf-e7ae-43e3-bf1c-ebbc6c1eb2c3 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:45.803515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125b1bf-e7ae-43e3-bf1c-ebbc6c1eb2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:45.869117 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-4125b1bf-e7ae-43e3-bf1c-ebbc6c1eb2c3 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:45.870314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4125b1bf-e7ae-43e3-bf1c-ebbc6c1eb2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:45.966528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 752/1480] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:46.264348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1c532-8467-45de-b0df-14ce74354719 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:46.265892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-05d1c532-8467-45de-b0df-14ce74354719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:46.276557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:46.411064 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1c532-8467-45de-b0df-14ce74354719 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:46.489056 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:46.520292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b17bbb-587d-4289-bea7-305e5b5cd620 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:46.521123 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b17bbb-587d-4289-bea7-305e5b5cd620 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:46.521596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b17bbb-587d-4289-bea7-305e5b5cd620 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:46.522113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b17bbb-587d-4289-bea7-305e5b5cd620 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:46.523730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-61b17bbb-587d-4289-bea7-305e5b5cd620 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:46.615813 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b17bbb-587d-4289-bea7-305e5b5cd620 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:46.617440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b17bbb-587d-4289-bea7-305e5b5cd620 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:46.635370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 729/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:50:48.965463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e4501-d85e-48c5-9d1f-90baa7f3cdd2 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:48.966031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e4501-d85e-48c5-9d1f-90baa7f3cdd2 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:48.966031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e4501-d85e-48c5-9d1f-90baa7f3cdd2 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:48.967396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 753/1482] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:48.973368 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01929b55-0eb3-433e-b159-65f39146c897 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:48.973800 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01929b55-0eb3-433e-b159-65f39146c897 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:48.973904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01929b55-0eb3-433e-b159-65f39146c897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:49.039209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-01929b55-0eb3-433e-b159-65f39146c897 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:49.040408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-01929b55-0eb3-433e-b159-65f39146c897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:49.086061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:49.137332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 730/1483] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:49.425386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e4501-d85e-48c5-9d1f-90baa7f3cdd2 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:49.426956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-030e4501-d85e-48c5-9d1f-90baa7f3cdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:49.449891 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:49.527838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e4501-d85e-48c5-9d1f-90baa7f3cdd2 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:49.599015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc8d4f-a911-46ca-85d0-d5856099b307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:49.599607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc8d4f-a911-46ca-85d0-d5856099b307 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:49.599884 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc8d4f-a911-46ca-85d0-d5856099b307 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:49.600143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc8d4f-a911-46ca-85d0-d5856099b307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:49.601719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-93bc8d4f-a911-46ca-85d0-d5856099b307 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:49.677340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:49.710989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc8d4f-a911-46ca-85d0-d5856099b307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:49.712479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc8d4f-a911-46ca-85d0-d5856099b307 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:49.729915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 754/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:50:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:50:51.947665 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751a464-d7bb-48df-80f6-55b7f40cdf83 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:51.948204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751a464-d7bb-48df-80f6-55b7f40cdf83 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:51.948204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751a464-d7bb-48df-80f6-55b7f40cdf83 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:51.949798 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 731/1485] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:51.955726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655f22a-be4c-444b-be12-07cf965e6e62 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:51.956188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655f22a-be4c-444b-be12-07cf965e6e62 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:51.956293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655f22a-be4c-444b-be12-07cf965e6e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:52.023524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-9655f22a-be4c-444b-be12-07cf965e6e62 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:52.024731 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9655f22a-be4c-444b-be12-07cf965e6e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:52.119977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 755/1486] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:52.410683 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751a464-d7bb-48df-80f6-55b7f40cdf83 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:52.412264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5751a464-d7bb-48df-80f6-55b7f40cdf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:52.514794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751a464-d7bb-48df-80f6-55b7f40cdf83 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:52.586297 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:52.587670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a21912-401d-4110-b718-0cd127511642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:52.588262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a21912-401d-4110-b718-0cd127511642 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:52.588536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a21912-401d-4110-b718-0cd127511642 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:52.588802 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a21912-401d-4110-b718-0cd127511642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:52.590409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-34a21912-401d-4110-b718-0cd127511642 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:52.701325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a21912-401d-4110-b718-0cd127511642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:52.702786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a21912-401d-4110-b718-0cd127511642 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:52.720501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 732/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:50:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:50:54.951193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49af4f-7ef8-432a-a8ca-a5295da95db8 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:54.951793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49af4f-7ef8-432a-a8ca-a5295da95db8 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:54.951793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49af4f-7ef8-432a-a8ca-a5295da95db8 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:54.953075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 756/1488] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:54.958876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4d894-647d-48be-9723-093ccf6d2d8f None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:54.959328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4d894-647d-48be-9723-093ccf6d2d8f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:54.959435 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4d894-647d-48be-9723-093ccf6d2d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:55.023557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-fda4d894-647d-48be-9723-093ccf6d2d8f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:55.024734 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-fda4d894-647d-48be-9723-093ccf6d2d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:55.077728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:55.122599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 733/1489] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:55.431889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49af4f-7ef8-432a-a8ca-a5295da95db8 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:55.433492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fb49af4f-7ef8-432a-a8ca-a5295da95db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:55.470875 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:55.537966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49af4f-7ef8-432a-a8ca-a5295da95db8 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:55.610989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed26b39-dcd3-47c3-b049-e52902696548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:55.611584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed26b39-dcd3-47c3-b049-e52902696548 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:55.611858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed26b39-dcd3-47c3-b049-e52902696548 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:55.612121 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed26b39-dcd3-47c3-b049-e52902696548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:55.613710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-bed26b39-dcd3-47c3-b049-e52902696548 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:55.695383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:55.724645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed26b39-dcd3-47c3-b049-e52902696548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:55.726069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed26b39-dcd3-47c3-b049-e52902696548 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:55.745203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 757/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:50:58.238965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f9c78-bf79-4471-8312-99db2e1aaf8a None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:58.239522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f9c78-bf79-4471-8312-99db2e1aaf8a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:58.239522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f9c78-bf79-4471-8312-99db2e1aaf8a None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:58.240824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 734/1491] 10.222.0.46 () {54 vars in 927 bytes} [Fri Sep 9 12:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:50:58.246316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ba2ab-778a-48cc-ba0d-dfa94ffe0d53 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:58.246807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ba2ab-778a-48cc-ba0d-dfa94ffe0d53 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:58.247051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ba2ab-778a-48cc-ba0d-dfa94ffe0d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:58.313833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-ca4ba2ab-778a-48cc-ba0d-dfa94ffe0d53 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:50:58.314969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ca4ba2ab-778a-48cc-ba0d-dfa94ffe0d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:58.409834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 758/1492] 10.222.0.46 () {58 vars in 1024 bytes} [Fri Sep 9 12:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:50:58.706031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f9c78-bf79-4471-8312-99db2e1aaf8a None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:50:58.707697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-407f9c78-bf79-4471-8312-99db2e1aaf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:58.809769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f9c78-bf79-4471-8312-99db2e1aaf8a None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:50:58.887372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2273a25-4040-4170-a3aa-a6aba9f706ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:50:58.887973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2273a25-4040-4170-a3aa-a6aba9f706ca service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:50:58.888249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2273a25-4040-4170-a3aa-a6aba9f706ca service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:50:58.888514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2273a25-4040-4170-a3aa-a6aba9f706ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:50:58.890126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b2273a25-4040-4170-a3aa-a6aba9f706ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:50:58.898769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:50:58.996011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2273a25-4040-4170-a3aa-a6aba9f706ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:50:58.997414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2273a25-4040-4170-a3aa-a6aba9f706ca service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:50:59.016514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 735/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:51:08.408358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=82 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:51:09.013588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=50 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:52:50.846061 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfa062-cb25-46a1-914e-a76142537568 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:50.846712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfa062-cb25-46a1-914e-a76142537568 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:50.846712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfa062-cb25-46a1-914e-a76142537568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:50.928190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-19bfa062-cb25-46a1-914e-a76142537568 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:52:50.929391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-19bfa062-cb25-46a1-914e-a76142537568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:52:51.032725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 759/1494] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:52:51.040682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61c415-d42d-49b1-979d-db978fece04f None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:51.041156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61c415-d42d-49b1-979d-db978fece04f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:51.041270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61c415-d42d-49b1-979d-db978fece04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:51.060641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-7f61c415-d42d-49b1-979d-db978fece04f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:52:51.061993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7f61c415-d42d-49b1-979d-db978fece04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:52:51.070606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 736/1495] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:52:51.075142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfa062-cb25-46a1-914e-a76142537568 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:52:51.085572 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfa062-cb25-46a1-914e-a76142537568 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:52:51.090115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907d56d-13cb-48e5-b32d-d787ab1608c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:52:51.090731 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907d56d-13cb-48e5-b32d-d787ab1608c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:51.091009 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907d56d-13cb-48e5-b32d-d787ab1608c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:51.091279 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907d56d-13cb-48e5-b32d-d787ab1608c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:51.091923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907d56d-13cb-48e5-b32d-d787ab1608c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:52:51.092221 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907d56d-13cb-48e5-b32d-d787ab1608c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:52:51.094339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: WARNING py.warnings [None req-b907d56d-13cb-48e5-b32d-d787ab1608c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:52:51.094339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: warnings.warn(msg) Sep 09 12:52:51.094339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: Sep 09 12:52:51.095246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907d56d-13cb-48e5-b32d-d787ab1608c5 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:52:51.106585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 760/1496] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:52:51.115112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61c415-d42d-49b1-979d-db978fece04f None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:52:51.124242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61c415-d42d-49b1-979d-db978fece04f None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:52:51.128820 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e592f-7e4e-47a7-8021-04bbe2c4df43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:52:51.129414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e592f-7e4e-47a7-8021-04bbe2c4df43 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:51.129688 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e592f-7e4e-47a7-8021-04bbe2c4df43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:51.129953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e592f-7e4e-47a7-8021-04bbe2c4df43 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:51.130607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e592f-7e4e-47a7-8021-04bbe2c4df43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:52:51.132326 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: WARNING py.warnings [None req-489e592f-7e4e-47a7-8021-04bbe2c4df43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:52:51.132326 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: warnings.warn(msg) Sep 09 12:52:51.132326 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: Sep 09 12:52:51.133498 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e592f-7e4e-47a7-8021-04bbe2c4df43 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:52:51.277049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 737/1497] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:52:51 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:52:51.281887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907d56d-13cb-48e5-b32d-d787ab1608c5 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:52:51.291770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907d56d-13cb-48e5-b32d-d787ab1608c5 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:52:51.341538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:52:51.353152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08c841-cff9-4ef1-9db2-89882e0ea0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:52:51.353762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08c841-cff9-4ef1-9db2-89882e0ea0cd admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:51.354029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08c841-cff9-4ef1-9db2-89882e0ea0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:51.354323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08c841-cff9-4ef1-9db2-89882e0ea0cd admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:51.354982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08c841-cff9-4ef1-9db2-89882e0ea0cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:52:51.355972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: WARNING py.warnings [None req-0a08c841-cff9-4ef1-9db2-89882e0ea0cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:52:51.355972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: warnings.warn(msg) Sep 09 12:52:51.355972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: Sep 09 12:52:51.357206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08c841-cff9-4ef1-9db2-89882e0ea0cd admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:52:51.457385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 761/1498] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:52:51 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:52:51.461435 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e592f-7e4e-47a7-8021-04bbe2c4df43 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:52:51.471705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e592f-7e4e-47a7-8021-04bbe2c4df43 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:52:51.476302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221d191-adff-472e-b84c-64e260a9c0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:52:51.476880 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221d191-adff-472e-b84c-64e260a9c0cf admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:51.477159 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221d191-adff-472e-b84c-64e260a9c0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:51.477430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221d191-adff-472e-b84c-64e260a9c0cf admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:51.478093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221d191-adff-472e-b84c-64e260a9c0cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:52:51.479140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: WARNING py.warnings [None req-8221d191-adff-472e-b84c-64e260a9c0cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:52:51.479140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: warnings.warn(msg) Sep 09 12:52:51.479140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: Sep 09 12:52:51.480018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221d191-adff-472e-b84c-64e260a9c0cf admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:52:51.492739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 738/1499] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:52:51.496556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08c841-cff9-4ef1-9db2-89882e0ea0cd admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:52:51.506767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08c841-cff9-4ef1-9db2-89882e0ea0cd admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:52:51.511386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0768e-37f9-4ce5-ace2-ec09dd8a5270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:52:51.511968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0768e-37f9-4ce5-ace2-ec09dd8a5270 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:51.512245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0768e-37f9-4ce5-ace2-ec09dd8a5270 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:51.512514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0768e-37f9-4ce5-ace2-ec09dd8a5270 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:51.513147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0768e-37f9-4ce5-ace2-ec09dd8a5270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:52:51.514137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: WARNING py.warnings [None req-97e0768e-37f9-4ce5-ace2-ec09dd8a5270 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:52:51.514137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: warnings.warn(msg) Sep 09 12:52:51.514137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: Sep 09 12:52:51.515636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0768e-37f9-4ce5-ace2-ec09dd8a5270 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:52:51.534823 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 762/1500] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:52:51.539194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221d191-adff-472e-b84c-64e260a9c0cf admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:52:51.549487 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221d191-adff-472e-b84c-64e260a9c0cf admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:52:51.554009 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a1c01-fc1d-43fa-aedc-babbc8b81809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:52:51.554683 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a1c01-fc1d-43fa-aedc-babbc8b81809 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:51.554964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a1c01-fc1d-43fa-aedc-babbc8b81809 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:51.555234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a1c01-fc1d-43fa-aedc-babbc8b81809 admin admin] PATH_INFO: `/v3/projects/17daad7c7d7a405ab26ce0a7a5c538c5/users/1e33bf35fdbb4843ae3f965e74751a56/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:51.569935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a1c01-fc1d-43fa-aedc-babbc8b81809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17daad7c7d7a405ab26ce0a7a5c538c5, user_id=1e33bf35fdbb4843ae3f965e74751a56, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:52:51.570982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: WARNING py.warnings [None req-799a1c01-fc1d-43fa-aedc-babbc8b81809 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:52:51.570982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: warnings.warn(msg) Sep 09 12:52:51.570982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: Sep 09 12:52:51.572870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a1c01-fc1d-43fa-aedc-babbc8b81809 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:52:51.662056 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 739/1501] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:52:51 2022] PUT /identity/v3/projects/17daad7c7d7a405ab26ce0a7a5c538c5/users/1e33bf35fdbb4843ae3f965e74751a56/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:52:51.675384 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1100b8b-f9be-42ab-91d6-dce43d803a4d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:51.675668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1100b8b-f9be-42ab-91d6-dce43d803a4d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:51.675939 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1100b8b-f9be-42ab-91d6-dce43d803a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:51.704809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-a1100b8b-f9be-42ab-91d6-dce43d803a4d None None] MFA Rules not processed for user `1e33bf35fdbb4843ae3f965e74751a56`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:52:51.706006 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a1100b8b-f9be-42ab-91d6-dce43d803a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:52:51.767620 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 763/1502] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:52:51.780813 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b1781-eaab-4d9a-a248-3edebbd1b187 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:51.781100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b1781-eaab-4d9a-a248-3edebbd1b187 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:51.781373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b1781-eaab-4d9a-a248-3edebbd1b187 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:51.782713 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 740/1503] 10.222.0.46 () {54 vars in 964 bytes} [Fri Sep 9 12:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:52:51.787374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce97d4-3732-49f7-bf10-48eb3721539f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:51.787654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce97d4-3732-49f7-bf10-48eb3721539f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:51.787921 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce97d4-3732-49f7-bf10-48eb3721539f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:51.846155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-6dce97d4-3732-49f7-bf10-48eb3721539f None None] MFA Rules not processed for user `002c117a85c1452e9fae80c755c6622f`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:52:51.847405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6dce97d4-3732-49f7-bf10-48eb3721539f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:52:51.933679 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 764/1504] 10.222.0.46 () {58 vars in 1061 bytes} [Fri Sep 9 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:52:51.942058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b1781-eaab-4d9a-a248-3edebbd1b187 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:52:51.952700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b1781-eaab-4d9a-a248-3edebbd1b187 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:52:51.956214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43350553-e551-4a20-a741-b5cdedadaabb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:52:51.956807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43350553-e551-4a20-a741-b5cdedadaabb service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:51.957089 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43350553-e551-4a20-a741-b5cdedadaabb service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:51.957364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43350553-e551-4a20-a741-b5cdedadaabb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:51.965358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43350553-e551-4a20-a741-b5cdedadaabb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:52:51.966795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43350553-e551-4a20-a741-b5cdedadaabb service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:52:51.975389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 741/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:52:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:52:52.022114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10588b5b-397b-453c-9cb3-fdbd5f458024 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:52.022395 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10588b5b-397b-453c-9cb3-fdbd5f458024 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:52.022721 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10588b5b-397b-453c-9cb3-fdbd5f458024 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:52.023971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 765/1506] 10.222.0.46 () {52 vars in 830 bytes} [Fri Sep 9 12:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:52:52.027520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43350553-e551-4a20-a741-b5cdedadaabb service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:52:52.088656 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43350553-e551-4a20-a741-b5cdedadaabb service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:52:52.093817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:52:52.094433 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:52.094743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:52.095016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] PATH_INFO: `/v3/users/1e33bf35fdbb4843ae3f965e74751a56` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:52.096120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e33bf35fdbb4843ae3f965e74751a56)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:52:52.097154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: WARNING py.warnings [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:52:52.097154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: warnings.warn(msg) Sep 09 12:52:52.097154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: Sep 09 12:52:52.098346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:52:52.299341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e33bf35fdbb4843ae3f965e74751a56'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:52:52.349027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e33bf35fdbb4843ae3f965e74751a56'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:52:52.349731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e33bf35fdbb4843ae3f965e74751a56'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:52:52.350357 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e33bf35fdbb4843ae3f965e74751a56'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:52:52.365162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e33bf35fdbb4843ae3f965e74751a56'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:52:52.387605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:52:52.395508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 742/1507] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:52:52 2022] DELETE /identity/v3/users/1e33bf35fdbb4843ae3f965e74751a56 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:52:52.399923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6d789-d31d-4650-9495-7a20fbb302bb None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:52.400211 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6d789-d31d-4650-9495-7a20fbb302bb None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:52.400488 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6d789-d31d-4650-9495-7a20fbb302bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:52.420125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-6da6d789-d31d-4650-9495-7a20fbb302bb None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:52:52.421932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6da6d789-d31d-4650-9495-7a20fbb302bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:52:52.430922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:52:52.479365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 766/1508] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:52:52.489184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:52:52.490649 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:52:52.567252 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3760a71-6116-4f46-a950-427f21a4e842 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:52:52.570843 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78600a18-392d-429d-a945-10a4fd5ef9ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:52:52.571448 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78600a18-392d-429d-a945-10a4fd5ef9ed service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:52.571727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78600a18-392d-429d-a945-10a4fd5ef9ed service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:52.571999 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78600a18-392d-429d-a945-10a4fd5ef9ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:52.581861 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78600a18-392d-429d-a945-10a4fd5ef9ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:52:52.583317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78600a18-392d-429d-a945-10a4fd5ef9ed service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:52:52.592512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 743/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:52:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:52:53.077540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6d789-d31d-4650-9495-7a20fbb302bb None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:52:53.079109 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6da6d789-d31d-4650-9495-7a20fbb302bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:52:53.091662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6d789-d31d-4650-9495-7a20fbb302bb None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:52:53.098279 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:52:53.099038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:52:53.099422 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:52:53.100181 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae admin admin] PATH_INFO: `/v3/projects/17daad7c7d7a405ab26ce0a7a5c538c5` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:52:53.100846 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae admin admin] RBAC: Authorizing `identity:delete_project(project_id=17daad7c7d7a405ab26ce0a7a5c538c5)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:52:53.106511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: WARNING py.warnings [None req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:52:53.106511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: warnings.warn(msg) Sep 09 12:52:53.106511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: Sep 09 12:52:53.106511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:52:53.236964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17daad7c7d7a405ab26ce0a7a5c538c5', 'request_id': 'req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:52:53.249978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17daad7c7d7a405ab26ce0a7a5c538c5', 'request_id': 'req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:52:53.303941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17daad7c7d7a405ab26ce0a7a5c538c5', 'request_id': 'req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:52:53.316356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae admin admin] The token cache is being invalidate because project 17daad7c7d7a405ab26ce0a7a5c538c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:52:53.320402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:52:53.320402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-798f9b74-326a-40b3-aa0e-1f4ed7e82cae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:52:53.320402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 767/1510] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:52:53 2022] DELETE /identity/v3/projects/17daad7c7d7a405ab26ce0a7a5c538c5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:53:02.588438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:53:03.313147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:54:25.284575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7326035-972d-4da0-8496-f939bb0a0afe None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:25.285198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7326035-972d-4da0-8496-f939bb0a0afe None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:25.285198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7326035-972d-4da0-8496-f939bb0a0afe None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:25.286602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:54:25.291348 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e79e2-a9d7-456f-bceb-9bf2093f8c56 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:25.291754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e79e2-a9d7-456f-bceb-9bf2093f8c56 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:25.291888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e79e2-a9d7-456f-bceb-9bf2093f8c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:25.313180 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-9c7e79e2-a9d7-456f-bceb-9bf2093f8c56 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:54:25.314365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9c7e79e2-a9d7-456f-bceb-9bf2093f8c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:25.371494 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:54:25.526185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7326035-972d-4da0-8496-f939bb0a0afe None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:54:25.527898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c7326035-972d-4da0-8496-f939bb0a0afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:25.630258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7326035-972d-4da0-8496-f939bb0a0afe None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:54:25.635670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832720b-9fba-45d1-ba7f-07f2e35f22b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:54:25.636288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832720b-9fba-45d1-ba7f-07f2e35f22b5 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:25.636566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832720b-9fba-45d1-ba7f-07f2e35f22b5 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:25.636852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832720b-9fba-45d1-ba7f-07f2e35f22b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:25.645851 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832720b-9fba-45d1-ba7f-07f2e35f22b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:54:25.647453 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832720b-9fba-45d1-ba7f-07f2e35f22b5 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:54:25.657653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 745/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:54:26.759214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f40eb-19c0-4030-843f-444e70547c3a None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:26.759760 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f40eb-19c0-4030-843f-444e70547c3a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:26.759760 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f40eb-19c0-4030-843f-444e70547c3a None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:26.761102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 769/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:54:26.767182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab6a6e-53d7-42d6-861e-b54f4ca25c56 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:26.767659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab6a6e-53d7-42d6-861e-b54f4ca25c56 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:26.767762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab6a6e-53d7-42d6-861e-b54f4ca25c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:26.787871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-06ab6a6e-53d7-42d6-861e-b54f4ca25c56 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:54:26.789057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-06ab6a6e-53d7-42d6-861e-b54f4ca25c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:26.798054 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 746/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:54:26.953530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f40eb-19c0-4030-843f-444e70547c3a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:54:26.968405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f40eb-19c0-4030-843f-444e70547c3a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:54:26.974361 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0b098-ad3d-4fc1-bfe2-6e2255940ac5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:54:26.975056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0b098-ad3d-4fc1-bfe2-6e2255940ac5 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:26.975338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0b098-ad3d-4fc1-bfe2-6e2255940ac5 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:26.975611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0b098-ad3d-4fc1-bfe2-6e2255940ac5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:26.983684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0b098-ad3d-4fc1-bfe2-6e2255940ac5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:54:26.985260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0b098-ad3d-4fc1-bfe2-6e2255940ac5 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:54:26.994826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 770/1516] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:54:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:54:27.168281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a0a2a-814a-47be-b409-3ebfd5772cb8 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:27.168569 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a0a2a-814a-47be-b409-3ebfd5772cb8 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:27.168846 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a0a2a-814a-47be-b409-3ebfd5772cb8 None None] PATH_INFO: `/v3` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:27.170114 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 747/1517] 10.222.0.15 () {54 vars in 925 bytes} [Fri Sep 9 12:54:27 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:54:27.176230 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1b9c1-95cd-4fbe-9003-ea76aa44b639 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:27.176514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1b9c1-95cd-4fbe-9003-ea76aa44b639 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:27.176786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1b9c1-95cd-4fbe-9003-ea76aa44b639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:27.225472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d7e1b9c1-95cd-4fbe-9003-ea76aa44b639 None None] MFA Rules not processed for user `33e177d86450489cacc86578ba26509d`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:54:27.226699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d7e1b9c1-95cd-4fbe-9003-ea76aa44b639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:27.296687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 771/1518] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Sep 9 12:54:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:54:31.131127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a46a2-279a-4b13-89ef-6495d8b83855 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:31.131603 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a46a2-279a-4b13-89ef-6495d8b83855 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:31.131718 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a46a2-279a-4b13-89ef-6495d8b83855 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:31.132957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 748/1519] 10.222.0.46 () {54 vars in 961 bytes} [Fri Sep 9 12:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:54:31.138002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796975c-5e48-4c5e-b570-e316a695e307 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:31.138429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796975c-5e48-4c5e-b570-e316a695e307 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:31.138572 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796975c-5e48-4c5e-b570-e316a695e307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:31.158820 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-5796975c-5e48-4c5e-b570-e316a695e307 None None] MFA Rules not processed for user `33e177d86450489cacc86578ba26509d`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:54:31.160006 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5796975c-5e48-4c5e-b570-e316a695e307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:31.168958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 772/1520] 10.222.0.46 () {58 vars in 1058 bytes} [Fri Sep 9 12:54:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:54:31.176588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a46a2-279a-4b13-89ef-6495d8b83855 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:54:31.190293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a46a2-279a-4b13-89ef-6495d8b83855 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:54:31.195264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317664e-929e-4b51-9fd4-8ac43d1687c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:54:31.195849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317664e-929e-4b51-9fd4-8ac43d1687c1 service placement] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:31.196129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317664e-929e-4b51-9fd4-8ac43d1687c1 service placement] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:31.196398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317664e-929e-4b51-9fd4-8ac43d1687c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:31.205638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d317664e-929e-4b51-9fd4-8ac43d1687c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:54:31.207242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d317664e-929e-4b51-9fd4-8ac43d1687c1 service placement] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:54:31.216593 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 749/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:54:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:54:38.483775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e6e8f-c745-4ae1-a21a-65bf99556053 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:38.484369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e6e8f-c745-4ae1-a21a-65bf99556053 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:38.484369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e6e8f-c745-4ae1-a21a-65bf99556053 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:38.485911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:54:38.493187 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fed6a5-07af-437f-8c32-9b13797a24c4 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:38.493709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fed6a5-07af-437f-8c32-9b13797a24c4 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:38.493814 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fed6a5-07af-437f-8c32-9b13797a24c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:38.514048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-89fed6a5-07af-437f-8c32-9b13797a24c4 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:54:38.515295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-89fed6a5-07af-437f-8c32-9b13797a24c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:38.523331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:54:38.675590 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e6e8f-c745-4ae1-a21a-65bf99556053 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:54:38.690396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e6e8f-c745-4ae1-a21a-65bf99556053 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:54:38.696040 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57876e0e-a272-4fd2-8c56-e2f56b401efa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:54:38.696623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57876e0e-a272-4fd2-8c56-e2f56b401efa service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:38.696897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57876e0e-a272-4fd2-8c56-e2f56b401efa service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:38.697163 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57876e0e-a272-4fd2-8c56-e2f56b401efa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:38.705406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57876e0e-a272-4fd2-8c56-e2f56b401efa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:54:38.706989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57876e0e-a272-4fd2-8c56-e2f56b401efa service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:54:38.716073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 774/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:54:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:54:39.793677 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a212a-2b86-4706-9d78-88b396c7f1f6 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:39.794217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a212a-2b86-4706-9d78-88b396c7f1f6 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:39.794217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a212a-2b86-4706-9d78-88b396c7f1f6 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:39.795617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:54:39.802724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaae5b-04ec-4cf5-905b-ae8c767b75fc None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:39.803147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaae5b-04ec-4cf5-905b-ae8c767b75fc None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:39.803248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaae5b-04ec-4cf5-905b-ae8c767b75fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:39.822625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-8adaae5b-04ec-4cf5-905b-ae8c767b75fc None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:54:39.823815 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8adaae5b-04ec-4cf5-905b-ae8c767b75fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:39.831955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:54:39.995675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a212a-2b86-4706-9d78-88b396c7f1f6 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:54:40.011076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a212a-2b86-4706-9d78-88b396c7f1f6 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:54:40.016711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d319a-e58d-4bf3-be5f-af62de0098d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:54:40.017452 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d319a-e58d-4bf3-be5f-af62de0098d7 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:40.017719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d319a-e58d-4bf3-be5f-af62de0098d7 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:40.017988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d319a-e58d-4bf3-be5f-af62de0098d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:40.026065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d319a-e58d-4bf3-be5f-af62de0098d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:54:40.027655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d319a-e58d-4bf3-be5f-af62de0098d7 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:54:40.036923 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 752/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:54:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:54:44.375148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed68895-61f8-4acd-9790-8fed218e7b63 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:44.375727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed68895-61f8-4acd-9790-8fed218e7b63 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:44.375727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed68895-61f8-4acd-9790-8fed218e7b63 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:44.377586 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:54:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:54:44.382947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561621d2-0bbf-4a83-acf1-a07013d4a805 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:44.383365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561621d2-0bbf-4a83-acf1-a07013d4a805 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:44.385109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561621d2-0bbf-4a83-acf1-a07013d4a805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:44.403626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-561621d2-0bbf-4a83-acf1-a07013d4a805 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:54:44.404865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-561621d2-0bbf-4a83-acf1-a07013d4a805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:44.413295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:54:44.700052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed68895-61f8-4acd-9790-8fed218e7b63 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:54:44.701994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0ed68895-61f8-4acd-9790-8fed218e7b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:44.730788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:54:44.758568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed68895-61f8-4acd-9790-8fed218e7b63 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:54:44.761978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad6719-22f4-426c-a95d-dd8a4b078e9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:54:44.762601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad6719-22f4-426c-a95d-dd8a4b078e9d service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:44.762876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad6719-22f4-426c-a95d-dd8a4b078e9d service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:44.763145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad6719-22f4-426c-a95d-dd8a4b078e9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:44.772709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad6719-22f4-426c-a95d-dd8a4b078e9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:54:44.774115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad6719-22f4-426c-a95d-dd8a4b078e9d service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:54:44.783929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 777/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:54:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:54:46.922156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c34680-6a99-4807-b578-23798420efe5 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:46.922837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c34680-6a99-4807-b578-23798420efe5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:46.922837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c34680-6a99-4807-b578-23798420efe5 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:46.925302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:54:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:54:46.930404 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811be4df-79c2-485f-8c4e-a2f5fd2e28e2 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:46.930833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811be4df-79c2-485f-8c4e-a2f5fd2e28e2 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:46.930993 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811be4df-79c2-485f-8c4e-a2f5fd2e28e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:46.950891 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-811be4df-79c2-485f-8c4e-a2f5fd2e28e2 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:54:46.952361 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-811be4df-79c2-485f-8c4e-a2f5fd2e28e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:46.961500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:54:47.259332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c34680-6a99-4807-b578-23798420efe5 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:54:47.270440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c34680-6a99-4807-b578-23798420efe5 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:54:47.273883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ffa61-7248-4a7c-983b-5baa6e52bba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:54:47.274471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ffa61-7248-4a7c-983b-5baa6e52bba2 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:47.274773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ffa61-7248-4a7c-983b-5baa6e52bba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:47.275043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ffa61-7248-4a7c-983b-5baa6e52bba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:47.283757 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ffa61-7248-4a7c-983b-5baa6e52bba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:54:47.285167 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ffa61-7248-4a7c-983b-5baa6e52bba2 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:54:47.294259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 755/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:54:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:54:51.124025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f10b9d-a7ce-4548-9c98-9c67ecca910a None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:51.124503 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f10b9d-a7ce-4548-9c98-9c67ecca910a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:51.124606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f10b9d-a7ce-4548-9c98-9c67ecca910a None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:51.125943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:54:51.131402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc56b4-8ccc-484c-9e0d-1aca056dca9a None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:51.131871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc56b4-8ccc-484c-9e0d-1aca056dca9a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:51.131978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc56b4-8ccc-484c-9e0d-1aca056dca9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:51.150816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-cfcc56b4-8ccc-484c-9e0d-1aca056dca9a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:54:51.151985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-cfcc56b4-8ccc-484c-9e0d-1aca056dca9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:51.160230 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:54:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:54:51.471508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f10b9d-a7ce-4548-9c98-9c67ecca910a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:54:51.473191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b2f10b9d-a7ce-4548-9c98-9c67ecca910a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:51.529333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f10b9d-a7ce-4548-9c98-9c67ecca910a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:54:51.532955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be134dc-7346-406a-b111-a1f31f4891dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:54:51.533551 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be134dc-7346-406a-b111-a1f31f4891dd service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:51.533829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be134dc-7346-406a-b111-a1f31f4891dd service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:51.534127 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be134dc-7346-406a-b111-a1f31f4891dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:51.542884 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be134dc-7346-406a-b111-a1f31f4891dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:54:51.544321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be134dc-7346-406a-b111-a1f31f4891dd service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:54:51.554088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 780/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:54:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:54:53.688396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f396d-950a-4c80-bcb5-1f74db37ee44 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:53.689609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f396d-950a-4c80-bcb5-1f74db37ee44 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:53.689609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f396d-950a-4c80-bcb5-1f74db37ee44 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:53.690982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:54:53.696266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610f681-b6dd-4f85-9003-6a68fb2a0316 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:53.696644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610f681-b6dd-4f85-9003-6a68fb2a0316 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:53.696809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610f681-b6dd-4f85-9003-6a68fb2a0316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:53.716235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d610f681-b6dd-4f85-9003-6a68fb2a0316 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:54:53.717407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d610f681-b6dd-4f85-9003-6a68fb2a0316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:53.725142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:54:54.033589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f396d-950a-4c80-bcb5-1f74db37ee44 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:54:54.035364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-814f396d-950a-4c80-bcb5-1f74db37ee44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:54:54.042871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=77 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:54:54.048539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f396d-950a-4c80-bcb5-1f74db37ee44 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:54:54.052024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb507c6-fa41-4734-91f4-698ab08fd640 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:54:54.055109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb507c6-fa41-4734-91f4-698ab08fd640 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:54:54.055109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb507c6-fa41-4734-91f4-698ab08fd640 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:54:54.055109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb507c6-fa41-4734-91f4-698ab08fd640 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:54:54.062014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb507c6-fa41-4734-91f4-698ab08fd640 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:54:54.063447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb507c6-fa41-4734-91f4-698ab08fd640 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:54:54.073308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 758/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:54:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:55:03.710901 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=47 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:55:04.095086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=5 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:19.458178 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e068c-800b-4229-b808-417e2da48132 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.462011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e068c-800b-4229-b808-417e2da48132 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.462011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e068c-800b-4229-b808-417e2da48132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.490330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-fb2e068c-800b-4229-b808-417e2da48132 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:19.491835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fb2e068c-800b-4229-b808-417e2da48132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:19.502289 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 782/1540] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:56:19.512720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5384492-85a8-4499-aa04-888735da89d4 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.513340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5384492-85a8-4499-aa04-888735da89d4 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.513340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5384492-85a8-4499-aa04-888735da89d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.545699 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-c5384492-85a8-4499-aa04-888735da89d4 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:19.547295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c5384492-85a8-4499-aa04-888735da89d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:19.558738 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 759/1541] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:56:19.563360 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e068c-800b-4229-b808-417e2da48132 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:19.580439 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e068c-800b-4229-b808-417e2da48132 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:19.590285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7af83-437c-49c9-8253-b46bdc57f490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:19.591213 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7af83-437c-49c9-8253-b46bdc57f490 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.591717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7af83-437c-49c9-8253-b46bdc57f490 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.592207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7af83-437c-49c9-8253-b46bdc57f490 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.593068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7af83-437c-49c9-8253-b46bdc57f490 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:19.593577 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7af83-437c-49c9-8253-b46bdc57f490 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:19.595391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7af83-437c-49c9-8253-b46bdc57f490 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:19.613291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8c5ae-38de-4872-a144-2bb6236507a2 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.613531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 783/1542] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:19.614166 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8c5ae-38de-4872-a144-2bb6236507a2 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.614670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8c5ae-38de-4872-a144-2bb6236507a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.632825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7af83-437c-49c9-8253-b46bdc57f490 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:19.653278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-2cd8c5ae-38de-4872-a144-2bb6236507a2 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:19.655040 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2cd8c5ae-38de-4872-a144-2bb6236507a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:19.657680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7af83-437c-49c9-8253-b46bdc57f490 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:19.667013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25beaf8-32a4-46e2-8b9e-4212e49e33ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:19.668269 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25beaf8-32a4-46e2-8b9e-4212e49e33ba admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.669329 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25beaf8-32a4-46e2-8b9e-4212e49e33ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.671051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25beaf8-32a4-46e2-8b9e-4212e49e33ba admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.672203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25beaf8-32a4-46e2-8b9e-4212e49e33ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:19.673730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: WARNING py.warnings [None req-d25beaf8-32a4-46e2-8b9e-4212e49e33ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:56:19.673730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: warnings.warn(msg) Sep 09 12:56:19.673730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: Sep 09 12:56:19.677880 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 760/1543] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:56:19.679097 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25beaf8-32a4-46e2-8b9e-4212e49e33ba admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:19.682304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f51782-9126-4b1d-94b8-3f5baa54b980 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.682853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f51782-9126-4b1d-94b8-3f5baa54b980 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.683303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f51782-9126-4b1d-94b8-3f5baa54b980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.711786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-99f51782-9126-4b1d-94b8-3f5baa54b980 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:19.713049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-99f51782-9126-4b1d-94b8-3f5baa54b980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:19.731512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 761/1544] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:19.734130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87134752-a643-43a4-b4ba-a08163aaa8f3 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.734130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87134752-a643-43a4-b4ba-a08163aaa8f3 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.734130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87134752-a643-43a4-b4ba-a08163aaa8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.761482 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-87134752-a643-43a4-b4ba-a08163aaa8f3 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:19.762796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-87134752-a643-43a4-b4ba-a08163aaa8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:19.774129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 762/1545] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:19.775399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b78e2-a07e-49e0-bf65-3bdd5a4a2f9d None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.775681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b78e2-a07e-49e0-bf65-3bdd5a4a2f9d None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.775952 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b78e2-a07e-49e0-bf65-3bdd5a4a2f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.805792 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-906b78e2-a07e-49e0-bf65-3bdd5a4a2f9d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:19.807534 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-906b78e2-a07e-49e0-bf65-3bdd5a4a2f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:19.815888 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 763/1546] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:19.817745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b3a07-672a-4ca6-b398-1d4b317999da None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.818031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b3a07-672a-4ca6-b398-1d4b317999da None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.818303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b3a07-672a-4ca6-b398-1d4b317999da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.826651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 784/1547] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/projects => generated 367 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:19.828459 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2c41e-3ae3-4de6-a821-d47eaaaf851d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.828747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2c41e-3ae3-4de6-a821-d47eaaaf851d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.829019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2c41e-3ae3-4de6-a821-d47eaaaf851d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.840556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ff8b3a07-672a-4ca6-b398-1d4b317999da None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:19.841740 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ff8b3a07-672a-4ca6-b398-1d4b317999da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:19.850645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-22d2c41e-3ae3-4de6-a821-d47eaaaf851d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:19.851824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-22d2c41e-3ae3-4de6-a821-d47eaaaf851d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:19.904147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 764/1548] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:19.905400 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16a0ae-a54b-46c1-8e2d-75ebc6d2d3a7 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.905692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16a0ae-a54b-46c1-8e2d-75ebc6d2d3a7 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.905990 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16a0ae-a54b-46c1-8e2d-75ebc6d2d3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.914313 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 785/1549] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:19.915692 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee82bb9-9b27-4743-b06b-6ddd2951261f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.915976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee82bb9-9b27-4743-b06b-6ddd2951261f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.916243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee82bb9-9b27-4743-b06b-6ddd2951261f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.925720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ba16a0ae-a54b-46c1-8e2d-75ebc6d2d3a7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:19.926954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ba16a0ae-a54b-46c1-8e2d-75ebc6d2d3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:19.934863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 765/1550] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:19.936033 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f989dc-e9f4-46f5-927f-8e77da68d173 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.936316 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f989dc-e9f4-46f5-927f-8e77da68d173 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.936590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f989dc-e9f4-46f5-927f-8e77da68d173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.939273 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-7ee82bb9-9b27-4743-b06b-6ddd2951261f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:19.940441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7ee82bb9-9b27-4743-b06b-6ddd2951261f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:19.948210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 786/1551] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:19.949042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee82bb9-9b27-4743-b06b-6ddd2951261f None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:19.959097 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee82bb9-9b27-4743-b06b-6ddd2951261f None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:19.959775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-44f989dc-e9f4-46f5-927f-8e77da68d173 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:19.960957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-44f989dc-e9f4-46f5-927f-8e77da68d173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:19.963694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040782c1-ffb9-4ac9-af74-17506bbeb612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:19.964219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040782c1-ffb9-4ac9-af74-17506bbeb612 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.964489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040782c1-ffb9-4ac9-af74-17506bbeb612 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.964753 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040782c1-ffb9-4ac9-af74-17506bbeb612 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.965385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040782c1-ffb9-4ac9-af74-17506bbeb612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:19.965676 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040782c1-ffb9-4ac9-af74-17506bbeb612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:19.967389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040782c1-ffb9-4ac9-af74-17506bbeb612 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:19.968993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 766/1552] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:19.969732 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f989dc-e9f4-46f5-927f-8e77da68d173 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:19.979381 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f989dc-e9f4-46f5-927f-8e77da68d173 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:19.983988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad61a1-df46-4a8b-8e3b-01faa41285e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:19.984579 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad61a1-df46-4a8b-8e3b-01faa41285e8 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.984866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad61a1-df46-4a8b-8e3b-01faa41285e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.985256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 787/1553] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:19.985372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad61a1-df46-4a8b-8e3b-01faa41285e8 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.985745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad61a1-df46-4a8b-8e3b-01faa41285e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:19.986374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07b6bc-70bd-4bec-bf36-199a313cda80 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:19.986685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07b6bc-70bd-4bec-bf36-199a313cda80 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:19.986925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: WARNING py.warnings [None req-7cad61a1-df46-4a8b-8e3b-01faa41285e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:56:19.986925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: warnings.warn(msg) Sep 09 12:56:19.986925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: Sep 09 12:56:19.987135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07b6bc-70bd-4bec-bf36-199a313cda80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:19.988704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad61a1-df46-4a8b-8e3b-01faa41285e8 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.006639 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0d07b6bc-70bd-4bec-bf36-199a313cda80 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:20.009415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0d07b6bc-70bd-4bec-bf36-199a313cda80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:20.015631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 788/1554] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:20.016425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07b6bc-70bd-4bec-bf36-199a313cda80 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.026429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07b6bc-70bd-4bec-bf36-199a313cda80 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.030945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2f7ef-3a2a-4eaa-a743-34f1bdf21bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.031513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2f7ef-3a2a-4eaa-a743-34f1bdf21bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.031785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2f7ef-3a2a-4eaa-a743-34f1bdf21bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.032052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2f7ef-3a2a-4eaa-a743-34f1bdf21bb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.032681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2f7ef-3a2a-4eaa-a743-34f1bdf21bb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:20.033003 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2f7ef-3a2a-4eaa-a743-34f1bdf21bb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.034641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2f7ef-3a2a-4eaa-a743-34f1bdf21bb0 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.045913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 767/1555] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:19 2022] POST /identity/v3/users => generated 284 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:20.045913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eed1fe-fa86-4d16-8467-5e863a3f8720 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.045913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eed1fe-fa86-4d16-8467-5e863a3f8720 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.045913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eed1fe-fa86-4d16-8467-5e863a3f8720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.051855 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 789/1556] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:20.051855 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2f7ef-3a2a-4eaa-a743-34f1bdf21bb0 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.063071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2f7ef-3a2a-4eaa-a743-34f1bdf21bb0 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.069016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122996b-5847-43be-8a5e-5725e20d95e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.069616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122996b-5847-43be-8a5e-5725e20d95e9 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.069897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122996b-5847-43be-8a5e-5725e20d95e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.070170 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122996b-5847-43be-8a5e-5725e20d95e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.070848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122996b-5847-43be-8a5e-5725e20d95e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:20.071153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122996b-5847-43be-8a5e-5725e20d95e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.071259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-07eed1fe-fa86-4d16-8467-5e863a3f8720 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:20.072482 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-07eed1fe-fa86-4d16-8467-5e863a3f8720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:20.072725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122996b-5847-43be-8a5e-5725e20d95e9 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.081286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 768/1557] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:20.083524 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cdafb-d240-470e-a5f6-89e7b30adf44 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.083992 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cdafb-d240-470e-a5f6-89e7b30adf44 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.084492 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cdafb-d240-470e-a5f6-89e7b30adf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.086913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 790/1558] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:20.088161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993d1eb-1994-41ff-9f96-34b38349f0ce None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.088460 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993d1eb-1994-41ff-9f96-34b38349f0ce None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.088743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993d1eb-1994-41ff-9f96-34b38349f0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.105684 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-a58cdafb-d240-470e-a5f6-89e7b30adf44 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:20.106889 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a58cdafb-d240-470e-a5f6-89e7b30adf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:20.109314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-2993d1eb-1994-41ff-9f96-34b38349f0ce None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:20.110501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2993d1eb-1994-41ff-9f96-34b38349f0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:20.115070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 769/1559] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:20.115979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cdafb-d240-470e-a5f6-89e7b30adf44 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.122010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 791/1560] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:20.122010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0ed88-c2b7-42a9-8cbe-b2401b737fa3 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.122010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0ed88-c2b7-42a9-8cbe-b2401b737fa3 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.122010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0ed88-c2b7-42a9-8cbe-b2401b737fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.127919 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cdafb-d240-470e-a5f6-89e7b30adf44 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.133116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e833b13-5ab7-463e-b8da-2cc56ee05403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.133718 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e833b13-5ab7-463e-b8da-2cc56ee05403 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.133997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e833b13-5ab7-463e-b8da-2cc56ee05403 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.134398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e833b13-5ab7-463e-b8da-2cc56ee05403 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.135068 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e833b13-5ab7-463e-b8da-2cc56ee05403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.136967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e833b13-5ab7-463e-b8da-2cc56ee05403 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.140448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-4fa0ed88-c2b7-42a9-8cbe-b2401b737fa3 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:20.141626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4fa0ed88-c2b7-42a9-8cbe-b2401b737fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:20.150306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 792/1561] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:20.151122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0ed88-c2b7-42a9-8cbe-b2401b737fa3 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.161103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0ed88-c2b7-42a9-8cbe-b2401b737fa3 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.165911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab468993-2b12-49b1-a2fe-5f7b88b3769d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.166565 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab468993-2b12-49b1-a2fe-5f7b88b3769d admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.166872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab468993-2b12-49b1-a2fe-5f7b88b3769d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.167142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab468993-2b12-49b1-a2fe-5f7b88b3769d admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.167780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab468993-2b12-49b1-a2fe-5f7b88b3769d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.169389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab468993-2b12-49b1-a2fe-5f7b88b3769d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.180163 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 793/1562] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:20.181009 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab468993-2b12-49b1-a2fe-5f7b88b3769d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.190620 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab468993-2b12-49b1-a2fe-5f7b88b3769d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.195794 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83434e-7e99-4674-a5e4-fdf053a0c016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.196367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83434e-7e99-4674-a5e4-fdf053a0c016 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.196647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83434e-7e99-4674-a5e4-fdf053a0c016 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.196921 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83434e-7e99-4674-a5e4-fdf053a0c016 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.197540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83434e-7e99-4674-a5e4-fdf053a0c016 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.199454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83434e-7e99-4674-a5e4-fdf053a0c016 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.216378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 770/1563] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:20.220401 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e833b13-5ab7-463e-b8da-2cc56ee05403 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.281377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e833b13-5ab7-463e-b8da-2cc56ee05403 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.286905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b64e39-233a-4461-ac2b-db21a1bc1ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.288586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b64e39-233a-4461-ac2b-db21a1bc1ffe admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.290319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b64e39-233a-4461-ac2b-db21a1bc1ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.290639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b64e39-233a-4461-ac2b-db21a1bc1ffe admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.292431 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b64e39-233a-4461-ac2b-db21a1bc1ffe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:20.292742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b64e39-233a-4461-ac2b-db21a1bc1ffe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.293761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: WARNING py.warnings [None req-b1b64e39-233a-4461-ac2b-db21a1bc1ffe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:56:20.293761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: warnings.warn(msg) Sep 09 12:56:20.293761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: Sep 09 12:56:20.294702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b64e39-233a-4461-ac2b-db21a1bc1ffe admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.297187 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:20.306661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 771/1564] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:20.312618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 794/1565] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:20.312618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83434e-7e99-4674-a5e4-fdf053a0c016 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.315278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b64e39-233a-4461-ac2b-db21a1bc1ffe admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.329992 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b64e39-233a-4461-ac2b-db21a1bc1ffe admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.337572 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:20.377649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83434e-7e99-4674-a5e4-fdf053a0c016 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.383067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97091099-53a0-4328-b304-6e7285970186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.383675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97091099-53a0-4328-b304-6e7285970186 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.383948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97091099-53a0-4328-b304-6e7285970186 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.384223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97091099-53a0-4328-b304-6e7285970186 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.390298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97091099-53a0-4328-b304-6e7285970186 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:20.390298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97091099-53a0-4328-b304-6e7285970186 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.390298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97091099-53a0-4328-b304-6e7285970186 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.390631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad61efe-5ef5-4f77-bbd5-c3b6041fab61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.390631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad61efe-5ef5-4f77-bbd5-c3b6041fab61 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.390631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad61efe-5ef5-4f77-bbd5-c3b6041fab61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.390631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad61efe-5ef5-4f77-bbd5-c3b6041fab61 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.391232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad61efe-5ef5-4f77-bbd5-c3b6041fab61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.393108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad61efe-5ef5-4f77-bbd5-c3b6041fab61 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.399236 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 795/1566] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:20.400157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97091099-53a0-4328-b304-6e7285970186 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.410852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97091099-53a0-4328-b304-6e7285970186 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.415094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475f3a9-9c98-49d0-b666-a3e150319c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.419015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475f3a9-9c98-49d0-b666-a3e150319c6e admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.419015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475f3a9-9c98-49d0-b666-a3e150319c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.419015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475f3a9-9c98-49d0-b666-a3e150319c6e admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.419015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475f3a9-9c98-49d0-b666-a3e150319c6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:20.419015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475f3a9-9c98-49d0-b666-a3e150319c6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.419015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475f3a9-9c98-49d0-b666-a3e150319c6e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.431139 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 796/1567] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:20.431982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475f3a9-9c98-49d0-b666-a3e150319c6e admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.441532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475f3a9-9c98-49d0-b666-a3e150319c6e admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.445879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a20d6-8e1c-4869-b875-831c8174e3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.446442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a20d6-8e1c-4869-b875-831c8174e3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.446760 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a20d6-8e1c-4869-b875-831c8174e3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.447240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a20d6-8e1c-4869-b875-831c8174e3b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.448154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a20d6-8e1c-4869-b875-831c8174e3b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:20.448887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a20d6-8e1c-4869-b875-831c8174e3b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.450427 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a20d6-8e1c-4869-b875-831c8174e3b1 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.462411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 797/1568] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:20.463205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a20d6-8e1c-4869-b875-831c8174e3b1 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.473143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a20d6-8e1c-4869-b875-831c8174e3b1 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.477689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323904b-f310-4889-aa41-8cf6b1b21464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.478251 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323904b-f310-4889-aa41-8cf6b1b21464 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.482379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323904b-f310-4889-aa41-8cf6b1b21464 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.482379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323904b-f310-4889-aa41-8cf6b1b21464 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.482379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323904b-f310-4889-aa41-8cf6b1b21464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.482379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323904b-f310-4889-aa41-8cf6b1b21464 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.491136 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 798/1569] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:20.491938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323904b-f310-4889-aa41-8cf6b1b21464 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.501068 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 772/1570] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:20.501321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323904b-f310-4889-aa41-8cf6b1b21464 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.502066 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad61efe-5ef5-4f77-bbd5-c3b6041fab61 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.560518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df29e62-26c2-46a8-acb5-344553e8b12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.561113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df29e62-26c2-46a8-acb5-344553e8b12f admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.561396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df29e62-26c2-46a8-acb5-344553e8b12f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.561659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df29e62-26c2-46a8-acb5-344553e8b12f admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.562276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df29e62-26c2-46a8-acb5-344553e8b12f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.564190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df29e62-26c2-46a8-acb5-344553e8b12f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.565215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad61efe-5ef5-4f77-bbd5-c3b6041fab61 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.570004 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3116a-1193-4b94-b975-dd8ff901b9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.570648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3116a-1193-4b94-b975-dd8ff901b9a1 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.570929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3116a-1193-4b94-b975-dd8ff901b9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.571201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3116a-1193-4b94-b975-dd8ff901b9a1 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.571826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3116a-1193-4b94-b975-dd8ff901b9a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.573710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3116a-1193-4b94-b975-dd8ff901b9a1 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.636476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 799/1571] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:20.637145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df29e62-26c2-46a8-acb5-344553e8b12f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.646623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df29e62-26c2-46a8-acb5-344553e8b12f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.651304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5231a-9250-4ead-980c-27073c4c3420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.651890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5231a-9250-4ead-980c-27073c4c3420 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.652167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5231a-9250-4ead-980c-27073c4c3420 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.652432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5231a-9250-4ead-980c-27073c4c3420 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.653046 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5231a-9250-4ead-980c-27073c4c3420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.655377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 773/1572] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/users => generated 295 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:20.655377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3116a-1193-4b94-b975-dd8ff901b9a1 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.661533 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5231a-9250-4ead-980c-27073c4c3420 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.667785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3116a-1193-4b94-b975-dd8ff901b9a1 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.672858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507288d-d72a-4ead-8500-d7e504f2b5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.673470 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507288d-d72a-4ead-8500-d7e504f2b5d2 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.673751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507288d-d72a-4ead-8500-d7e504f2b5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.674023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507288d-d72a-4ead-8500-d7e504f2b5d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.674684 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507288d-d72a-4ead-8500-d7e504f2b5d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.677183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507288d-d72a-4ead-8500-d7e504f2b5d2 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.738443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 800/1573] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:20.738443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5231a-9250-4ead-980c-27073c4c3420 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.755900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 774/1574] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:20.755900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507288d-d72a-4ead-8500-d7e504f2b5d2 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.784136 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:20.801472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5231a-9250-4ead-980c-27073c4c3420 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.803765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:20.816718 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507288d-d72a-4ead-8500-d7e504f2b5d2 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:20.821492 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa73b37-0d66-4595-b536-868b315b547a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.822157 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa73b37-0d66-4595-b536-868b315b547a admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.824577 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa73b37-0d66-4595-b536-868b315b547a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.824577 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa73b37-0d66-4595-b536-868b315b547a admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.824577 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa73b37-0d66-4595-b536-868b315b547a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.825443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa73b37-0d66-4595-b536-868b315b547a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.855978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2b9a1-3e9a-4ca7-b74b-020b650fc3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:20.856578 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2b9a1-3e9a-4ca7-b74b-020b650fc3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:20.856852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2b9a1-3e9a-4ca7-b74b-020b650fc3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:20.857118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2b9a1-3e9a-4ca7-b74b-020b650fc3e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:20.857739 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2b9a1-3e9a-4ca7-b74b-020b650fc3e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:20.859643 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2b9a1-3e9a-4ca7-b74b-020b650fc3e3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:20.912699 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 775/1575] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/projects => generated 357 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:20.913587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa73b37-0d66-4595-b536-868b315b547a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.945446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 801/1576] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/projects => generated 379 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:20.945446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2b9a1-3e9a-4ca7-b74b-020b650fc3e3 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:20.974726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa73b37-0d66-4595-b536-868b315b547a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.003861 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2b9a1-3e9a-4ca7-b74b-020b650fc3e3 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.008564 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6275b-036c-4dee-85b8-53e53c831b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.009347 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6275b-036c-4dee-85b8-53e53c831b45 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.009751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6275b-036c-4dee-85b8-53e53c831b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.010024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6275b-036c-4dee-85b8-53e53c831b45 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.010685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6275b-036c-4dee-85b8-53e53c831b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.012615 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6275b-036c-4dee-85b8-53e53c831b45 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.031190 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba862af-ed41-48f5-b84d-b0109e4cd255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.031845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba862af-ed41-48f5-b84d-b0109e4cd255 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.032123 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba862af-ed41-48f5-b84d-b0109e4cd255 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.032393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba862af-ed41-48f5-b84d-b0109e4cd255 admin admin] PATH_INFO: `/v3/projects/99de43adb16f490fa1ab22985b27c1c3/users/6445de29c36d4f8f8ef51d2277e287ef/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.046012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba862af-ed41-48f5-b84d-b0109e4cd255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99de43adb16f490fa1ab22985b27c1c3, user_id=6445de29c36d4f8f8ef51d2277e287ef, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.048642 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba862af-ed41-48f5-b84d-b0109e4cd255 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.123076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 802/1577] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:20 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:21.124184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6275b-036c-4dee-85b8-53e53c831b45 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.134308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6275b-036c-4dee-85b8-53e53c831b45 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.135371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 776/1578] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:20 2022] PUT /identity/v3/projects/99de43adb16f490fa1ab22985b27c1c3/users/6445de29c36d4f8f8ef51d2277e287ef/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:21.139927 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba862af-ed41-48f5-b84d-b0109e4cd255 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.192668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c0294-c087-4cec-b790-21725f189ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.193279 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c0294-c087-4cec-b790-21725f189ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.193557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c0294-c087-4cec-b790-21725f189ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.193825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c0294-c087-4cec-b790-21725f189ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.194466 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c0294-c087-4cec-b790-21725f189ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.196065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c0294-c087-4cec-b790-21725f189ee9 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.206255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 803/1579] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.207091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c0294-c087-4cec-b790-21725f189ee9 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.208182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba862af-ed41-48f5-b84d-b0109e4cd255 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.212011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:21.213110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093fbb49-ee98-4a21-a822-5a14abcfd62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.213704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fbb49-ee98-4a21-a822-5a14abcfd62a admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.213981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fbb49-ee98-4a21-a822-5a14abcfd62a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.214249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fbb49-ee98-4a21-a822-5a14abcfd62a admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.215119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093fbb49-ee98-4a21-a822-5a14abcfd62a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.224954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093fbb49-ee98-4a21-a822-5a14abcfd62a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.225107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c0294-c087-4cec-b790-21725f189ee9 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.225107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a691dd-4e39-4a40-a3ab-e07b219fe685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.225306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:21.225397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a691dd-4e39-4a40-a3ab-e07b219fe685 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.225397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a691dd-4e39-4a40-a3ab-e07b219fe685 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.225397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a691dd-4e39-4a40-a3ab-e07b219fe685 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.225397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a691dd-4e39-4a40-a3ab-e07b219fe685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.225397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a691dd-4e39-4a40-a3ab-e07b219fe685 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.227097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 777/1580] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.230930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093fbb49-ee98-4a21-a822-5a14abcfd62a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.241290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093fbb49-ee98-4a21-a822-5a14abcfd62a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.245849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29add7e-7895-4e4f-a2b5-7e4c97d35105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.247873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29add7e-7895-4e4f-a2b5-7e4c97d35105 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.247873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29add7e-7895-4e4f-a2b5-7e4c97d35105 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.247873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29add7e-7895-4e4f-a2b5-7e4c97d35105 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.247873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29add7e-7895-4e4f-a2b5-7e4c97d35105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.249573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29add7e-7895-4e4f-a2b5-7e4c97d35105 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.323616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 804/1581] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:21 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:21.324348 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a691dd-4e39-4a40-a3ab-e07b219fe685 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.335074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a691dd-4e39-4a40-a3ab-e07b219fe685 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.338723 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 778/1582] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:21 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:21.338723 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29add7e-7895-4e4f-a2b5-7e4c97d35105 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.340146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f21c72-b42f-4727-ae0a-7e2373d22aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.340737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f21c72-b42f-4727-ae0a-7e2373d22aef admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.341042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f21c72-b42f-4727-ae0a-7e2373d22aef admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.341310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f21c72-b42f-4727-ae0a-7e2373d22aef admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.341931 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f21c72-b42f-4727-ae0a-7e2373d22aef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.343900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f21c72-b42f-4727-ae0a-7e2373d22aef admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.346142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29add7e-7895-4e4f-a2b5-7e4c97d35105 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.356063 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b61f4-b08e-4be3-a083-24f861292833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.356678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b61f4-b08e-4be3-a083-24f861292833 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.356964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b61f4-b08e-4be3-a083-24f861292833 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.357246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b61f4-b08e-4be3-a083-24f861292833 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.357878 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b61f4-b08e-4be3-a083-24f861292833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.360609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b61f4-b08e-4be3-a083-24f861292833 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.408273 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 805/1583] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:21 2022] POST /identity/v3/users => generated 279 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:21.409133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f21c72-b42f-4727-ae0a-7e2373d22aef admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.419516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f21c72-b42f-4727-ae0a-7e2373d22aef admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.424464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a7396-702a-413a-8b45-bf9dbe6b9f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.425051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a7396-702a-413a-8b45-bf9dbe6b9f95 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.425333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a7396-702a-413a-8b45-bf9dbe6b9f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.425604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a7396-702a-413a-8b45-bf9dbe6b9f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.426242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a7396-702a-413a-8b45-bf9dbe6b9f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.426659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 779/1584] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:21 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:21.427858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a7396-702a-413a-8b45-bf9dbe6b9f95 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.431786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4b3de-a269-48ae-b854-8ac90bd51c61 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.431786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4b3de-a269-48ae-b854-8ac90bd51c61 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.431786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4b3de-a269-48ae-b854-8ac90bd51c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.441957 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 806/1585] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.441957 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a7396-702a-413a-8b45-bf9dbe6b9f95 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.449560 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a7396-702a-413a-8b45-bf9dbe6b9f95 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.454768 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b18413-48a2-452e-ad92-b79c0987c69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.455575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b18413-48a2-452e-ad92-b79c0987c69e admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.455947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b18413-48a2-452e-ad92-b79c0987c69e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.456318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b18413-48a2-452e-ad92-b79c0987c69e admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.457997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b18413-48a2-452e-ad92-b79c0987c69e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.458724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-60d4b3de-a269-48ae-b854-8ac90bd51c61 None None] MFA Rules not processed for user `6445de29c36d4f8f8ef51d2277e287ef`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:21.459831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b18413-48a2-452e-ad92-b79c0987c69e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.459959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-60d4b3de-a269-48ae-b854-8ac90bd51c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:21.474355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 807/1586] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.474355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b18413-48a2-452e-ad92-b79c0987c69e admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.481663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b18413-48a2-452e-ad92-b79c0987c69e admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.486416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e2eff-d536-4f7b-8833-cb0245d3fde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.487014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e2eff-d536-4f7b-8833-cb0245d3fde4 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.487395 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e2eff-d536-4f7b-8833-cb0245d3fde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.487686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e2eff-d536-4f7b-8833-cb0245d3fde4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.488317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e2eff-d536-4f7b-8833-cb0245d3fde4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.489858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e2eff-d536-4f7b-8833-cb0245d3fde4 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.499727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 808/1587] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.500499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e2eff-d536-4f7b-8833-cb0245d3fde4 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.510417 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e2eff-d536-4f7b-8833-cb0245d3fde4 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.514970 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc875f-634c-4fd2-858b-5bc890aae4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.515530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc875f-634c-4fd2-858b-5bc890aae4de admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.515802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc875f-634c-4fd2-858b-5bc890aae4de admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.516074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc875f-634c-4fd2-858b-5bc890aae4de admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.516686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc875f-634c-4fd2-858b-5bc890aae4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.518260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc875f-634c-4fd2-858b-5bc890aae4de admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.528678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 809/1588] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.533159 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 780/1589] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:21.533314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc875f-634c-4fd2-858b-5bc890aae4de admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.533431 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4b3de-a269-48ae-b854-8ac90bd51c61 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.542120 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc875f-634c-4fd2-858b-5bc890aae4de admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.544109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4b3de-a269-48ae-b854-8ac90bd51c61 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.546699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987da07-4b7f-44ca-82ab-b729897a429f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.547282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987da07-4b7f-44ca-82ab-b729897a429f admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.547553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987da07-4b7f-44ca-82ab-b729897a429f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.547818 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987da07-4b7f-44ca-82ab-b729897a429f admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.548443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987da07-4b7f-44ca-82ab-b729897a429f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.548881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddfd36-0b1f-4c2d-bd35-592a61b6f917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.549478 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfd36-0b1f-4c2d-bd35-592a61b6f917 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.549760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfd36-0b1f-4c2d-bd35-592a61b6f917 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.549989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987da07-4b7f-44ca-82ab-b729897a429f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.550082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfd36-0b1f-4c2d-bd35-592a61b6f917 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.550709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddfd36-0b1f-4c2d-bd35-592a61b6f917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.552340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddfd36-0b1f-4c2d-bd35-592a61b6f917 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.560753 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 810/1590] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.564568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 781/1591] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.564568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddfd36-0b1f-4c2d-bd35-592a61b6f917 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.564890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987da07-4b7f-44ca-82ab-b729897a429f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.573860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddfd36-0b1f-4c2d-bd35-592a61b6f917 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.574621 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987da07-4b7f-44ca-82ab-b729897a429f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.578670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3dbea-118a-443e-9238-5e16610d5fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.579249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3dbea-118a-443e-9238-5e16610d5fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.579526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3dbea-118a-443e-9238-5e16610d5fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.579796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3dbea-118a-443e-9238-5e16610d5fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.580045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6162a-a01a-407a-9007-6f3ae30e58c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.580413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3dbea-118a-443e-9238-5e16610d5fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.580638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6162a-a01a-407a-9007-6f3ae30e58c0 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.580919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6162a-a01a-407a-9007-6f3ae30e58c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.581192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6162a-a01a-407a-9007-6f3ae30e58c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.582552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6162a-a01a-407a-9007-6f3ae30e58c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.582704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3dbea-118a-443e-9238-5e16610d5fa8 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.584118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6162a-a01a-407a-9007-6f3ae30e58c0 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.592396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 782/1592] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.593231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3dbea-118a-443e-9238-5e16610d5fa8 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.596999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 811/1593] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.596999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6162a-a01a-407a-9007-6f3ae30e58c0 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.603528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3dbea-118a-443e-9238-5e16610d5fa8 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.605943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6162a-a01a-407a-9007-6f3ae30e58c0 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.608324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ea0f0-fcf5-489c-b6e9-b2f8db63a575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.609070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ea0f0-fcf5-489c-b6e9-b2f8db63a575 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.609447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ea0f0-fcf5-489c-b6e9-b2f8db63a575 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.609783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ea0f0-fcf5-489c-b6e9-b2f8db63a575 admin admin] PATH_INFO: `/v3/projects/3a03f8b7f17247c18fa91a7d64a98eb3/users/0c8d72e1a3054b4db0a79e72a361c0be/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.610746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5ffc4-fa68-4413-9d42-c82cd29f3456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.611372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5ffc4-fa68-4413-9d42-c82cd29f3456 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.611652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5ffc4-fa68-4413-9d42-c82cd29f3456 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.611928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5ffc4-fa68-4413-9d42-c82cd29f3456 admin admin] PATH_INFO: `/v3/projects/8880382269b94daa8a25f92178a478d5/users/6ec951604cde4768beb02906caf6d7c2/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.624662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ea0f0-fcf5-489c-b6e9-b2f8db63a575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a03f8b7f17247c18fa91a7d64a98eb3, user_id=0c8d72e1a3054b4db0a79e72a361c0be, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.626609 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5ffc4-fa68-4413-9d42-c82cd29f3456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8880382269b94daa8a25f92178a478d5, user_id=6ec951604cde4768beb02906caf6d7c2, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.627319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ea0f0-fcf5-489c-b6e9-b2f8db63a575 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.627836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: WARNING py.warnings [None req-6fb5ffc4-fa68-4413-9d42-c82cd29f3456 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:56:21.627836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: warnings.warn(msg) Sep 09 12:56:21.627836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: Sep 09 12:56:21.630849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5ffc4-fa68-4413-9d42-c82cd29f3456 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.689059 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 783/1594] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:21 2022] PUT /identity/v3/projects/3a03f8b7f17247c18fa91a7d64a98eb3/users/0c8d72e1a3054b4db0a79e72a361c0be/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:21.689890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ea0f0-fcf5-489c-b6e9-b2f8db63a575 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.714200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 812/1595] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:21 2022] PUT /identity/v3/projects/8880382269b94daa8a25f92178a478d5/users/6ec951604cde4768beb02906caf6d7c2/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:21.715312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5ffc4-fa68-4413-9d42-c82cd29f3456 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.754581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ea0f0-fcf5-489c-b6e9-b2f8db63a575 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.766257 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:21.770543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5ffc4-fa68-4413-9d42-c82cd29f3456 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.774181 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b97ff-74ba-4d39-84a0-df5fa9e7b55a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.774849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b97ff-74ba-4d39-84a0-df5fa9e7b55a service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.775843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b97ff-74ba-4d39-84a0-df5fa9e7b55a service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.775843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b97ff-74ba-4d39-84a0-df5fa9e7b55a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.812928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077fef0-1e48-47a0-b450-c2d161c34b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.813532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077fef0-1e48-47a0-b450-c2d161c34b9e admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.813807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077fef0-1e48-47a0-b450-c2d161c34b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.814111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077fef0-1e48-47a0-b450-c2d161c34b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.814790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077fef0-1e48-47a0-b450-c2d161c34b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.816421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077fef0-1e48-47a0-b450-c2d161c34b9e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.823809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:21.826318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 784/1596] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.827368 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077fef0-1e48-47a0-b450-c2d161c34b9e admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.836752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b97ff-74ba-4d39-84a0-df5fa9e7b55a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.837640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077fef0-1e48-47a0-b450-c2d161c34b9e admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.838198 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b97ff-74ba-4d39-84a0-df5fa9e7b55a service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.842398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c09e58-79a8-4ee5-833e-3141b27a6926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.843052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09e58-79a8-4ee5-833e-3141b27a6926 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.843334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09e58-79a8-4ee5-833e-3141b27a6926 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.843605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09e58-79a8-4ee5-833e-3141b27a6926 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.844228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c09e58-79a8-4ee5-833e-3141b27a6926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.845818 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c09e58-79a8-4ee5-833e-3141b27a6926 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.847594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 813/1597] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:21.848484 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b97ff-74ba-4d39-84a0-df5fa9e7b55a service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.856919 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 785/1598] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.857716 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c09e58-79a8-4ee5-833e-3141b27a6926 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.861941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b97ff-74ba-4d39-84a0-df5fa9e7b55a service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.867443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c09e58-79a8-4ee5-833e-3141b27a6926 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.867951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e615a-9a00-4dd6-9e7b-26fa6668aed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.868805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e615a-9a00-4dd6-9e7b-26fa6668aed1 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.869523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e615a-9a00-4dd6-9e7b-26fa6668aed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.870119 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e615a-9a00-4dd6-9e7b-26fa6668aed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.871211 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e615a-9a00-4dd6-9e7b-26fa6668aed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.872581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76de793-5284-4253-8f86-46598d5305d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.873222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76de793-5284-4253-8f86-46598d5305d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.873553 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76de793-5284-4253-8f86-46598d5305d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.873835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76de793-5284-4253-8f86-46598d5305d5 admin admin] PATH_INFO: `/v3/projects/ceed6a1f0f4542cbafa4b2a46b16d5fb/users/63a2902f465f4fd1aed80060c2669259/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.874135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e615a-9a00-4dd6-9e7b-26fa6668aed1 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.887936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 814/1599] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.889118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e615a-9a00-4dd6-9e7b-26fa6668aed1 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:21.890079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76de793-5284-4253-8f86-46598d5305d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceed6a1f0f4542cbafa4b2a46b16d5fb, user_id=63a2902f465f4fd1aed80060c2669259, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.892732 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76de793-5284-4253-8f86-46598d5305d5 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.900148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e615a-9a00-4dd6-9e7b-26fa6668aed1 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:21.904771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65670b03-a7c9-480b-96ca-93a77b8c17f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:21.905354 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65670b03-a7c9-480b-96ca-93a77b8c17f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.905632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65670b03-a7c9-480b-96ca-93a77b8c17f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.905905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65670b03-a7c9-480b-96ca-93a77b8c17f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.906566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65670b03-a7c9-480b-96ca-93a77b8c17f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:21.908131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65670b03-a7c9-480b-96ca-93a77b8c17f3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:21.918786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 815/1600] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:21.924040 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc70bf4-9f75-4565-b569-ab71a259b71e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.924040 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc70bf4-9f75-4565-b569-ab71a259b71e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.924040 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc70bf4-9f75-4565-b569-ab71a259b71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.950581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-8bc70bf4-9f75-4565-b569-ab71a259b71e None None] MFA Rules not processed for user `0c8d72e1a3054b4db0a79e72a361c0be`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:21.952214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8bc70bf4-9f75-4565-b569-ab71a259b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:21.961570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 786/1601] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:21 2022] PUT /identity/v3/projects/ceed6a1f0f4542cbafa4b2a46b16d5fb/users/63a2902f465f4fd1aed80060c2669259/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:21.961570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b9d90-b9ce-4c46-a2e4-b6f74e29f6cb None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:21.961570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b9d90-b9ce-4c46-a2e4-b6f74e29f6cb None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:21.961570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b9d90-b9ce-4c46-a2e4-b6f74e29f6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:21.980314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-b04b9d90-b9ce-4c46-a2e4-b6f74e29f6cb None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:21.982037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b04b9d90-b9ce-4c46-a2e4-b6f74e29f6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:22.027672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 816/1602] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:22.028680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc70bf4-9f75-4565-b569-ab71a259b71e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:22.065195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 787/1603] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:22.066682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b9d90-b9ce-4c46-a2e4-b6f74e29f6cb None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:22.082758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b9d90-b9ce-4c46-a2e4-b6f74e29f6cb None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:22.090415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca2d69-a594-4859-a4a5-c47117286743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:22.091474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca2d69-a594-4859-a4a5-c47117286743 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.091910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca2d69-a594-4859-a4a5-c47117286743 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.092148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca2d69-a594-4859-a4a5-c47117286743 admin admin] PATH_INFO: `/v3/projects/7df3b6aedfe14bd1b8263b75027f80b1/users/849c51110cfa47e0b2b5de048c6991ac/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.109259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca2d69-a594-4859-a4a5-c47117286743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df3b6aedfe14bd1b8263b75027f80b1, user_id=849c51110cfa47e0b2b5de048c6991ac, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:22.112239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca2d69-a594-4859-a4a5-c47117286743 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:22.118962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc70bf4-9f75-4565-b569-ab71a259b71e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:22.124823 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b163be-eece-406e-a4f3-54c4a6884acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:22.125498 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b163be-eece-406e-a4f3-54c4a6884acc admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.125942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b163be-eece-406e-a4f3-54c4a6884acc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.126220 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b163be-eece-406e-a4f3-54c4a6884acc admin admin] PATH_INFO: `/v3/projects/fbb40c611f584fa6a4658e9159bc9351/users/c83cc26f922442b68fea25edd3501248/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.143384 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b163be-eece-406e-a4f3-54c4a6884acc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb40c611f584fa6a4658e9159bc9351, user_id=c83cc26f922442b68fea25edd3501248, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:22.146206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b163be-eece-406e-a4f3-54c4a6884acc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:22.181049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 788/1604] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:22 2022] PUT /identity/v3/projects/7df3b6aedfe14bd1b8263b75027f80b1/users/849c51110cfa47e0b2b5de048c6991ac/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:22.182204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca2d69-a594-4859-a4a5-c47117286743 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:22.199003 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 817/1605] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:22 2022] PUT /identity/v3/projects/fbb40c611f584fa6a4658e9159bc9351/users/c83cc26f922442b68fea25edd3501248/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:22.203749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b163be-eece-406e-a4f3-54c4a6884acc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:22.249065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:22.250466 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:22.259284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca2d69-a594-4859-a4a5-c47117286743 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:22.264043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e802a-af78-4833-a368-f4774996d2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:22.264737 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e802a-af78-4833-a368-f4774996d2a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.265020 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e802a-af78-4833-a368-f4774996d2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.265300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e802a-af78-4833-a368-f4774996d2a6 admin admin] PATH_INFO: `/v3/projects/9073b1c2abbb43999ae2d9fc7023d338/users/eee8eee033184e9d9fc3f6cb8a802a70/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.267871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b163be-eece-406e-a4f3-54c4a6884acc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:22.273080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946b038-b799-415f-b669-78c993bd914b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:22.278171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946b038-b799-415f-b669-78c993bd914b admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.278171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946b038-b799-415f-b669-78c993bd914b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.278171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946b038-b799-415f-b669-78c993bd914b admin admin] PATH_INFO: `/v3/projects/9c2a31fedb6843ba8ffea570a374acf5/users/c71eb53fc47c4f82add6fdd3d469230e/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.280285 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e802a-af78-4833-a368-f4774996d2a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9073b1c2abbb43999ae2d9fc7023d338, user_id=eee8eee033184e9d9fc3f6cb8a802a70, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:22.283030 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e802a-af78-4833-a368-f4774996d2a6 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:22.289174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946b038-b799-415f-b669-78c993bd914b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c2a31fedb6843ba8ffea570a374acf5, user_id=c71eb53fc47c4f82add6fdd3d469230e, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:22.291874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946b038-b799-415f-b669-78c993bd914b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:22.339930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 789/1606] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:22 2022] PUT /identity/v3/projects/9073b1c2abbb43999ae2d9fc7023d338/users/eee8eee033184e9d9fc3f6cb8a802a70/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:22.341235 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd14e3-2425-4220-b82f-c49be8d396c1 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.341556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd14e3-2425-4220-b82f-c49be8d396c1 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.341795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd14e3-2425-4220-b82f-c49be8d396c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.363829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-c8cd14e3-2425-4220-b82f-c49be8d396c1 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:22.365011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c8cd14e3-2425-4220-b82f-c49be8d396c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:22.374537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 818/1607] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:22 2022] PUT /identity/v3/projects/9c2a31fedb6843ba8ffea570a374acf5/users/c71eb53fc47c4f82add6fdd3d469230e/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:22.376038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd4213-ad48-4e6f-b3f0-8813aef10d93 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.376145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd4213-ad48-4e6f-b3f0-8813aef10d93 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.376343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd4213-ad48-4e6f-b3f0-8813aef10d93 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.377728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 819/1608] 10.222.0.46 () {54 vars in 962 bytes} [Fri Sep 9 12:56:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:56:22.379106 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd4213-ad48-4e6f-b3f0-8813aef10d93 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:22.435551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 790/1609] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:22.436887 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66556724-4b97-45c6-8f9a-ecd6544a813f None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.437179 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66556724-4b97-45c6-8f9a-ecd6544a813f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.437457 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66556724-4b97-45c6-8f9a-ecd6544a813f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.438057 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd4213-ad48-4e6f-b3f0-8813aef10d93 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:22.442800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2c3c4-95e9-4cbf-86f0-9b6ecfe9ab13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:22.443403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2c3c4-95e9-4cbf-86f0-9b6ecfe9ab13 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.443682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2c3c4-95e9-4cbf-86f0-9b6ecfe9ab13 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.443969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2c3c4-95e9-4cbf-86f0-9b6ecfe9ab13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.467643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-66556724-4b97-45c6-8f9a-ecd6544a813f None None] MFA Rules not processed for user `849c51110cfa47e0b2b5de048c6991ac`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:22.469161 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-66556724-4b97-45c6-8f9a-ecd6544a813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:22.511662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2c3c4-95e9-4cbf-86f0-9b6ecfe9ab13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:22.512834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2c3c4-95e9-4cbf-86f0-9b6ecfe9ab13 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:22.522607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 820/1610] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:22.523834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d0c57-12b5-42c2-a362-72b8d9d45089 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.524116 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d0c57-12b5-42c2-a362-72b8d9d45089 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.524390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d0c57-12b5-42c2-a362-72b8d9d45089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.540481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 791/1611] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:22.540481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dfaa7-5213-46c9-8729-406ed7643dc5 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.540481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dfaa7-5213-46c9-8729-406ed7643dc5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.540481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dfaa7-5213-46c9-8729-406ed7643dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.556870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-eb1d0c57-12b5-42c2-a362-72b8d9d45089 None None] MFA Rules not processed for user `c83cc26f922442b68fea25edd3501248`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:22.559524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-eb1d0c57-12b5-42c2-a362-72b8d9d45089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:22.568596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ce4dfaa7-5213-46c9-8729-406ed7643dc5 None None] MFA Rules not processed for user `eee8eee033184e9d9fc3f6cb8a802a70`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:22.569775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ce4dfaa7-5213-46c9-8729-406ed7643dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:22.627887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 821/1612] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:22.630079 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b8dd8-b1a4-40ef-a261-7ede65097d47 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.630356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b8dd8-b1a4-40ef-a261-7ede65097d47 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.630652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b8dd8-b1a4-40ef-a261-7ede65097d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.634034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:22.638614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:22.643396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 792/1613] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:22.648704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72930a3-fe37-4e38-894f-dbcaef089480 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.648997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72930a3-fe37-4e38-894f-dbcaef089480 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.653537 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72930a3-fe37-4e38-894f-dbcaef089480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.653664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-100b8dd8-b1a4-40ef-a261-7ede65097d47 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:22.653664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-100b8dd8-b1a4-40ef-a261-7ede65097d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:22.662622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 822/1614] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:22.663701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b8dd8-b1a4-40ef-a261-7ede65097d47 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:22.674589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b8dd8-b1a4-40ef-a261-7ede65097d47 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:22.681034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0c950-8375-4a62-b7c6-7998da90d110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:22.681034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0c950-8375-4a62-b7c6-7998da90d110 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.681034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0c950-8375-4a62-b7c6-7998da90d110 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.681034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0c950-8375-4a62-b7c6-7998da90d110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.697241 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0c950-8375-4a62-b7c6-7998da90d110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:22.698903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0c950-8375-4a62-b7c6-7998da90d110 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:22.703282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-d72930a3-fe37-4e38-894f-dbcaef089480 None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:22.705086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d72930a3-fe37-4e38-894f-dbcaef089480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:22.732412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 823/1615] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:22.734776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0c950-8375-4a62-b7c6-7998da90d110 service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:22.748954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0c950-8375-4a62-b7c6-7998da90d110 service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:22.758927 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918f860-2491-4064-b486-93aa283fef5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:22.760143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918f860-2491-4064-b486-93aa283fef5f admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.760474 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918f860-2491-4064-b486-93aa283fef5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.760753 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918f860-2491-4064-b486-93aa283fef5f admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.761876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918f860-2491-4064-b486-93aa283fef5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:22.764281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918f860-2491-4064-b486-93aa283fef5f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:22.793074 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 793/1616] 10.222.0.46 () {58 vars in 1059 bytes} [Fri Sep 9 12:56:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:22.793910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72930a3-fe37-4e38-894f-dbcaef089480 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:22.822829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72930a3-fe37-4e38-894f-dbcaef089480 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:22.839053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00470c05-3264-45ae-818a-5813103a0dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:22.839053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00470c05-3264-45ae-818a-5813103a0dfb admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:22.839053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00470c05-3264-45ae-818a-5813103a0dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:22.839053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00470c05-3264-45ae-818a-5813103a0dfb admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:22.839390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00470c05-3264-45ae-818a-5813103a0dfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:22.839390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00470c05-3264-45ae-818a-5813103a0dfb admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:22.911112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 824/1617] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:22 2022] POST /identity/v3/projects => generated 363 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:22.918970 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918f860-2491-4064-b486-93aa283fef5f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:22.963774 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 794/1618] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:22 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:22.964847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00470c05-3264-45ae-818a-5813103a0dfb admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:22.974459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-00470c05-3264-45ae-818a-5813103a0dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:23.020524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918f860-2491-4064-b486-93aa283fef5f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:23.085834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00470c05-3264-45ae-818a-5813103a0dfb admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:23.090355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b56ea-e8b4-46a2-82f0-b161d9b29ad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:23.091159 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b56ea-e8b4-46a2-82f0-b161d9b29ad5 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:23.091508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b56ea-e8b4-46a2-82f0-b161d9b29ad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:23.092106 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b56ea-e8b4-46a2-82f0-b161d9b29ad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:23.138681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398cc4f-9e72-4da4-a320-817460ed5730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:23.141527 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398cc4f-9e72-4da4-a320-817460ed5730 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:23.141906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398cc4f-9e72-4da4-a320-817460ed5730 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:23.142415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398cc4f-9e72-4da4-a320-817460ed5730 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:23.144333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398cc4f-9e72-4da4-a320-817460ed5730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:23.147054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398cc4f-9e72-4da4-a320-817460ed5730 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:23.201116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b56ea-e8b4-46a2-82f0-b161d9b29ad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:23.203554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b56ea-e8b4-46a2-82f0-b161d9b29ad5 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:23.218746 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 795/1619] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:23.219710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b56ea-e8b4-46a2-82f0-b161d9b29ad5 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:23.228172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:23.263761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 825/1620] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:22 2022] POST /identity/v3/projects => generated 355 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:23.264807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398cc4f-9e72-4da4-a320-817460ed5730 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:23.356052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b56ea-e8b4-46a2-82f0-b161d9b29ad5 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:23.381353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:23.388092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398cc4f-9e72-4da4-a320-817460ed5730 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:23.396438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f3a48-e169-477e-b0c1-ec4c30d6b34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:23.398206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f3a48-e169-477e-b0c1-ec4c30d6b34a admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:23.398965 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f3a48-e169-477e-b0c1-ec4c30d6b34a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:23.399486 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f3a48-e169-477e-b0c1-ec4c30d6b34a admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:23.400790 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f3a48-e169-477e-b0c1-ec4c30d6b34a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:23.403369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f3a48-e169-477e-b0c1-ec4c30d6b34a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:23.445712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b701a-cb5e-4548-98ff-fcb396e85eb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:23.446329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b701a-cb5e-4548-98ff-fcb396e85eb6 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:23.446660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b701a-cb5e-4548-98ff-fcb396e85eb6 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:23.446970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b701a-cb5e-4548-98ff-fcb396e85eb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:23.511805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 826/1621] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:23 2022] POST /identity/v3/users => generated 288 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:23.513919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f3a48-e169-477e-b0c1-ec4c30d6b34a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:23.524430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b701a-cb5e-4548-98ff-fcb396e85eb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:23.525803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f3a48-e169-477e-b0c1-ec4c30d6b34a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:23.526106 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b701a-cb5e-4548-98ff-fcb396e85eb6 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:23.531816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5edbb-6bed-470f-86c4-7190a2467c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:23.543184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5edbb-6bed-470f-86c4-7190a2467c91 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:23.543184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5edbb-6bed-470f-86c4-7190a2467c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:23.543184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5edbb-6bed-470f-86c4-7190a2467c91 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:23.543184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5edbb-6bed-470f-86c4-7190a2467c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:23.543184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5edbb-6bed-470f-86c4-7190a2467c91 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:23.543756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 796/1622] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:23.543756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b701a-cb5e-4548-98ff-fcb396e85eb6 service nova] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:23.556977 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b701a-cb5e-4548-98ff-fcb396e85eb6 service nova] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:23.562619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a1532-196c-412f-b62d-503902c7b041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:23.563245 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a1532-196c-412f-b62d-503902c7b041 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:23.563518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a1532-196c-412f-b62d-503902c7b041 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:23.563796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a1532-196c-412f-b62d-503902c7b041 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:23.564816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a1532-196c-412f-b62d-503902c7b041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:23.566761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a1532-196c-412f-b62d-503902c7b041 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:23.687869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 827/1623] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:23 2022] POST /identity/v3/users => generated 299 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:23.689749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5edbb-6bed-470f-86c4-7190a2467c91 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:23.710913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5edbb-6bed-470f-86c4-7190a2467c91 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:23.723736 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 797/1624] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:23 2022] POST /identity/v3/users => generated 284 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:23.724865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a1532-196c-412f-b62d-503902c7b041 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:23.727667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59406c-4255-4b29-97c3-118e9babef1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:23.729428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59406c-4255-4b29-97c3-118e9babef1e admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:23.731255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59406c-4255-4b29-97c3-118e9babef1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:23.733660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59406c-4255-4b29-97c3-118e9babef1e admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:23.742589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59406c-4255-4b29-97c3-118e9babef1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:23.745038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59406c-4255-4b29-97c3-118e9babef1e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:23.749702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a1532-196c-412f-b62d-503902c7b041 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:23.758017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4674e-fe0c-420f-a0bb-b9612db4f015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:23.758830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4674e-fe0c-420f-a0bb-b9612db4f015 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:23.759141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4674e-fe0c-420f-a0bb-b9612db4f015 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:23.759479 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4674e-fe0c-420f-a0bb-b9612db4f015 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:23.760235 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4674e-fe0c-420f-a0bb-b9612db4f015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:23.761839 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4674e-fe0c-420f-a0bb-b9612db4f015 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:23.784138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 828/1625] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:23.790411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59406c-4255-4b29-97c3-118e9babef1e admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:23.805286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 798/1626] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:23.806372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4674e-fe0c-420f-a0bb-b9612db4f015 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:23.823719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59406c-4255-4b29-97c3-118e9babef1e admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:23.831423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabacd4-62c0-4a93-b7bb-5e74713e7a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:23.831664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabacd4-62c0-4a93-b7bb-5e74713e7a5e admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:23.831914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabacd4-62c0-4a93-b7bb-5e74713e7a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:23.832138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabacd4-62c0-4a93-b7bb-5e74713e7a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:23.834461 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabacd4-62c0-4a93-b7bb-5e74713e7a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:23.835765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabacd4-62c0-4a93-b7bb-5e74713e7a5e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:23.839535 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4674e-fe0c-420f-a0bb-b9612db4f015 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:23.855747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 829/1627] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:23.856913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabacd4-62c0-4a93-b7bb-5e74713e7a5e admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:23.866619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d99c5-0f68-40f5-9ffb-949ef4ae1e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:23.867379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d99c5-0f68-40f5-9ffb-949ef4ae1e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:23.867638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d99c5-0f68-40f5-9ffb-949ef4ae1e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:23.867965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d99c5-0f68-40f5-9ffb-949ef4ae1e0f admin admin] PATH_INFO: `/v3/projects/015d08a67b8a4d509263b36b2a4ffb06/users/c57df58aea51481192560ce8ec324703/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:23.876851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabacd4-62c0-4a93-b7bb-5e74713e7a5e admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:23.883038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f164a5a-537f-44f0-8f3e-fb9bf49be48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:23.883950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f164a5a-537f-44f0-8f3e-fb9bf49be48d admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:23.884381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f164a5a-537f-44f0-8f3e-fb9bf49be48d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:23.884725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f164a5a-537f-44f0-8f3e-fb9bf49be48d admin admin] PATH_INFO: `/v3/projects/1002db0c193e41629578594fa645df2d/users/68f4c80a7fb44dc4a70445f4ed6c7845/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:23.887432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:23.918152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f164a5a-537f-44f0-8f3e-fb9bf49be48d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1002db0c193e41629578594fa645df2d, user_id=68f4c80a7fb44dc4a70445f4ed6c7845, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:23.920634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d99c5-0f68-40f5-9ffb-949ef4ae1e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=015d08a67b8a4d509263b36b2a4ffb06, user_id=c57df58aea51481192560ce8ec324703, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:23.923399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d99c5-0f68-40f5-9ffb-949ef4ae1e0f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:23.923792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f164a5a-537f-44f0-8f3e-fb9bf49be48d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:24.014874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 830/1628] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:23 2022] PUT /identity/v3/projects/1002db0c193e41629578594fa645df2d/users/68f4c80a7fb44dc4a70445f4ed6c7845/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:24.016411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f164a5a-537f-44f0-8f3e-fb9bf49be48d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:24.045494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 799/1629] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:23 2022] PUT /identity/v3/projects/015d08a67b8a4d509263b36b2a4ffb06/users/c57df58aea51481192560ce8ec324703/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:24.047824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec1f82-5a36-4b96-9a92-9a90be38486f None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.048443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec1f82-5a36-4b96-9a92-9a90be38486f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.048725 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec1f82-5a36-4b96-9a92-9a90be38486f None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.051191 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 800/1630] 10.222.0.46 () {54 vars in 912 bytes} [Fri Sep 9 12:56:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:56:24.052235 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec1f82-5a36-4b96-9a92-9a90be38486f None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:24.164443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f164a5a-537f-44f0-8f3e-fb9bf49be48d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:24.168991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec1f82-5a36-4b96-9a92-9a90be38486f None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:24.174848 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8de8b9-c36c-4dd0-aa69-41997422d990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:24.179893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8de8b9-c36c-4dd0-aa69-41997422d990 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.179893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8de8b9-c36c-4dd0-aa69-41997422d990 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.179893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8de8b9-c36c-4dd0-aa69-41997422d990 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.179893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8de8b9-c36c-4dd0-aa69-41997422d990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:24.179893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8de8b9-c36c-4dd0-aa69-41997422d990 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:24.187091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a1fff-4cf5-4efc-b8ae-bb236cd734ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:24.187201 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a1fff-4cf5-4efc-b8ae-bb236cd734ca admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.187462 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a1fff-4cf5-4efc-b8ae-bb236cd734ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.188223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a1fff-4cf5-4efc-b8ae-bb236cd734ca admin admin] PATH_INFO: `/v3/projects/1bf77868b1b84c7bb97e5ecd9348b00b/users/6c24badd5d4844ae96312710474a8443/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.199354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 801/1631] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:24.209082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8de8b9-c36c-4dd0-aa69-41997422d990 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:24.213466 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a1fff-4cf5-4efc-b8ae-bb236cd734ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf77868b1b84c7bb97e5ecd9348b00b, user_id=6c24badd5d4844ae96312710474a8443, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:24.214287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8de8b9-c36c-4dd0-aa69-41997422d990 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:24.217057 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a1fff-4cf5-4efc-b8ae-bb236cd734ca admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:24.221805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf114aa8-aedf-4562-afb9-dc9162884765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:24.222548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf114aa8-aedf-4562-afb9-dc9162884765 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.222900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf114aa8-aedf-4562-afb9-dc9162884765 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.223463 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf114aa8-aedf-4562-afb9-dc9162884765 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.226287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf114aa8-aedf-4562-afb9-dc9162884765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:24.226287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf114aa8-aedf-4562-afb9-dc9162884765 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:24.242554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 802/1632] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:24.242554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479f5ad-19fe-4421-89d3-942bee3891c7 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.242554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479f5ad-19fe-4421-89d3-942bee3891c7 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.242554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479f5ad-19fe-4421-89d3-942bee3891c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.267541 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-4479f5ad-19fe-4421-89d3-942bee3891c7 None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:24.269706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-4479f5ad-19fe-4421-89d3-942bee3891c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:24.355893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 831/1633] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:24 2022] PUT /identity/v3/projects/1bf77868b1b84c7bb97e5ecd9348b00b/users/6c24badd5d4844ae96312710474a8443/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:24.361327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a1fff-4cf5-4efc-b8ae-bb236cd734ca admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:24.399009 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 803/1634] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 12:56:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:24.400327 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c31fa-f0a5-4a89-b7c7-be38cae85074 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.400815 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c31fa-f0a5-4a89-b7c7-be38cae85074 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.401330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c31fa-f0a5-4a89-b7c7-be38cae85074 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.414655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 804/1635] 10.222.0.46 () {54 vars in 962 bytes} [Fri Sep 9 12:56:24 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:56:24.415835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c31fa-f0a5-4a89-b7c7-be38cae85074 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:24.631425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a1fff-4cf5-4efc-b8ae-bb236cd734ca admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:24.631425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5380732-b38e-45b4-a678-271b0239db3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:24.631425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5380732-b38e-45b4-a678-271b0239db3c admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.631425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5380732-b38e-45b4-a678-271b0239db3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.632839 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5380732-b38e-45b4-a678-271b0239db3c admin admin] PATH_INFO: `/v3/projects/1002db0c193e41629578594fa645df2d/users/68f4c80a7fb44dc4a70445f4ed6c7845/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.632839 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5380732-b38e-45b4-a678-271b0239db3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1002db0c193e41629578594fa645df2d, user_id=68f4c80a7fb44dc4a70445f4ed6c7845, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:24.632839 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5380732-b38e-45b4-a678-271b0239db3c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:24.632839 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:24.632839 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 832/1636] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:24 2022] PUT /identity/v3/projects/1002db0c193e41629578594fa645df2d/users/68f4c80a7fb44dc4a70445f4ed6c7845/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:24.633267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5380732-b38e-45b4-a678-271b0239db3c admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:24.633406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c31fa-f0a5-4a89-b7c7-be38cae85074 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:24.633406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:24.633541 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5380732-b38e-45b4-a678-271b0239db3c admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:24.633541 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f0139-b083-4b2f-a611-8c512c890d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:24.633541 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f0139-b083-4b2f-a611-8c512c890d13 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.633541 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f0139-b083-4b2f-a611-8c512c890d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.634154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f0139-b083-4b2f-a611-8c512c890d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.634154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f0139-b083-4b2f-a611-8c512c890d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:24.634154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f0139-b083-4b2f-a611-8c512c890d13 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:24.634462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177110d-5984-4c4a-841b-20f6faa4ce78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:24.634462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177110d-5984-4c4a-841b-20f6faa4ce78 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.634462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177110d-5984-4c4a-841b-20f6faa4ce78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.634462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177110d-5984-4c4a-841b-20f6faa4ce78 admin admin] PATH_INFO: `/v3/projects/015d08a67b8a4d509263b36b2a4ffb06/users/c57df58aea51481192560ce8ec324703/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.634882 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177110d-5984-4c4a-841b-20f6faa4ce78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=015d08a67b8a4d509263b36b2a4ffb06, user_id=c57df58aea51481192560ce8ec324703, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:24.634882 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177110d-5984-4c4a-841b-20f6faa4ce78 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:24.635091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 833/1637] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:24.635091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf77328-ff12-432b-bd2d-13b55017e46d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.635091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf77328-ff12-432b-bd2d-13b55017e46d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.635091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf77328-ff12-432b-bd2d-13b55017e46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.635567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 805/1638] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:24 2022] PUT /identity/v3/projects/015d08a67b8a4d509263b36b2a4ffb06/users/c57df58aea51481192560ce8ec324703/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:24.635567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177110d-5984-4c4a-841b-20f6faa4ce78 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:24.635738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-adf77328-ff12-432b-bd2d-13b55017e46d None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:24.635738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-adf77328-ff12-432b-bd2d-13b55017e46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:24.664820 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177110d-5984-4c4a-841b-20f6faa4ce78 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:24.669532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c52ab7-3ff5-42b6-86f2-d8a510cca705 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:24.670586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c52ab7-3ff5-42b6-86f2-d8a510cca705 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.670880 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c52ab7-3ff5-42b6-86f2-d8a510cca705 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.671199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c52ab7-3ff5-42b6-86f2-d8a510cca705 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.713599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 834/1639] 10.222.0.46 () {58 vars in 1059 bytes} [Fri Sep 9 12:56:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:24.715179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcd330-cf5d-4ba1-bb6c-655fdc152ca4 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.715495 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcd330-cf5d-4ba1-bb6c-655fdc152ca4 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.715775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcd330-cf5d-4ba1-bb6c-655fdc152ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.747174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-1dfcd330-cf5d-4ba1-bb6c-655fdc152ca4 None None] MFA Rules not processed for user `68f4c80a7fb44dc4a70445f4ed6c7845`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:24.748407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1dfcd330-cf5d-4ba1-bb6c-655fdc152ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:24.755760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c52ab7-3ff5-42b6-86f2-d8a510cca705 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:24.758062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c52ab7-3ff5-42b6-86f2-d8a510cca705 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:24.769855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 806/1640] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:24.770897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c52ab7-3ff5-42b6-86f2-d8a510cca705 service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:24.837850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 835/1641] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:24.844906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f11b1-d325-45aa-b7d2-885d7365a23a None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.844906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f11b1-d325-45aa-b7d2-885d7365a23a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.844906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f11b1-d325-45aa-b7d2-885d7365a23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.868869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c52ab7-3ff5-42b6-86f2-d8a510cca705 service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:24.874116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e02f4-fe45-4b14-b257-6c32ebb9c06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:24.875011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e02f4-fe45-4b14-b257-6c32ebb9c06a admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.875419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e02f4-fe45-4b14-b257-6c32ebb9c06a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.875880 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e02f4-fe45-4b14-b257-6c32ebb9c06a admin admin] PATH_INFO: `/v3/projects/1bf77868b1b84c7bb97e5ecd9348b00b/users/6c24badd5d4844ae96312710474a8443/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.878062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e02f4-fe45-4b14-b257-6c32ebb9c06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf77868b1b84c7bb97e5ecd9348b00b, user_id=6c24badd5d4844ae96312710474a8443, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:24.879750 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-813f11b1-d325-45aa-b7d2-885d7365a23a None None] MFA Rules not processed for user `c57df58aea51481192560ce8ec324703`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:24.881123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-813f11b1-d325-45aa-b7d2-885d7365a23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:24.881544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e02f4-fe45-4b14-b257-6c32ebb9c06a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:24.894872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 807/1642] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:24 2022] PUT /identity/v3/projects/1bf77868b1b84c7bb97e5ecd9348b00b/users/6c24badd5d4844ae96312710474a8443/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:24.897161 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f5c70-f297-4f54-b97e-86d3c815524a None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.897589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f5c70-f297-4f54-b97e-86d3c815524a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.898049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f5c70-f297-4f54-b97e-86d3c815524a None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:24.907834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 808/1643] 10.222.0.46 () {54 vars in 912 bytes} [Fri Sep 9 12:56:24 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:56:24.908975 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f5c70-f297-4f54-b97e-86d3c815524a None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:24.973782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 836/1644] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:24.976146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a840fe0-2997-4aaf-9539-0a8ef19ebc71 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:24.978563 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a840fe0-2997-4aaf-9539-0a8ef19ebc71 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:24.978967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a840fe0-2997-4aaf-9539-0a8ef19ebc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:25.046774 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f5c70-f297-4f54-b97e-86d3c815524a None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:25.048025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0a840fe0-2997-4aaf-9539-0a8ef19ebc71 None None] MFA Rules not processed for user `6c24badd5d4844ae96312710474a8443`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:25.052327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0a840fe0-2997-4aaf-9539-0a8ef19ebc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:25.054960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69254803-8959-436c-9a2c-bf043278e197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:25.055597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69254803-8959-436c-9a2c-bf043278e197 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:25.055931 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69254803-8959-436c-9a2c-bf043278e197 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:25.056212 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69254803-8959-436c-9a2c-bf043278e197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:25.162318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 837/1645] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:24 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:25.163786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42531-0cf5-48f4-976d-3a915b987bce None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:25.164149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42531-0cf5-48f4-976d-3a915b987bce None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:25.164672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42531-0cf5-48f4-976d-3a915b987bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:25.169626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69254803-8959-436c-9a2c-bf043278e197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:25.171183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69254803-8959-436c-9a2c-bf043278e197 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:25.185089 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 809/1646] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:25.225708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-6ea42531-0cf5-48f4-976d-3a915b987bce None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:25.227353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6ea42531-0cf5-48f4-976d-3a915b987bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:25.318868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69254803-8959-436c-9a2c-bf043278e197 service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:25.329976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69254803-8959-436c-9a2c-bf043278e197 service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:25.335104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeaf31c-1318-4c97-abbb-33d19cac244c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:25.336576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaf31c-1318-4c97-abbb-33d19cac244c service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:25.337434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaf31c-1318-4c97-abbb-33d19cac244c service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:25.337672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaf31c-1318-4c97-abbb-33d19cac244c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:25.377165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 838/1647] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 12:56:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:25.409044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeaf31c-1318-4c97-abbb-33d19cac244c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:25.411201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeaf31c-1318-4c97-abbb-33d19cac244c service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:25.423662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 810/1648] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:56:25.639664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42531-0cf5-48f4-976d-3a915b987bce None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:25.664039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42531-0cf5-48f4-976d-3a915b987bce None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:25.679094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9c95f-d589-415e-bfad-2addb6eb28b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:25.679094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9c95f-d589-415e-bfad-2addb6eb28b4 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:25.680389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9c95f-d589-415e-bfad-2addb6eb28b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:25.681132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9c95f-d589-415e-bfad-2addb6eb28b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:25.695439 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9c95f-d589-415e-bfad-2addb6eb28b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:25.697064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9c95f-d589-415e-bfad-2addb6eb28b4 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:25.736268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 839/1649] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:25.737126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeaf31c-1318-4c97-abbb-33d19cac244c service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:25.760748 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9c95f-d589-415e-bfad-2addb6eb28b4 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:25.771330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9c95f-d589-415e-bfad-2addb6eb28b4 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:25.777983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c48a1-0172-4ccc-b689-3eef47b9460a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:25.782711 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c48a1-0172-4ccc-b689-3eef47b9460a service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:25.783176 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c48a1-0172-4ccc-b689-3eef47b9460a service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:25.783577 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c48a1-0172-4ccc-b689-3eef47b9460a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:25.810233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c48a1-0172-4ccc-b689-3eef47b9460a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:25.813982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c48a1-0172-4ccc-b689-3eef47b9460a service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:25.857131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 840/1650] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:56:25.910933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeaf31c-1318-4c97-abbb-33d19cac244c service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:25.933726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc275b-2120-4b40-8dbd-f4523b9262f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:25.933726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc275b-2120-4b40-8dbd-f4523b9262f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:25.933726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc275b-2120-4b40-8dbd-f4523b9262f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:25.933726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc275b-2120-4b40-8dbd-f4523b9262f4 admin admin] PATH_INFO: `/v3/users/c83cc26f922442b68fea25edd3501248` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:25.933983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc275b-2120-4b40-8dbd-f4523b9262f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c83cc26f922442b68fea25edd3501248)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:25.933983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc275b-2120-4b40-8dbd-f4523b9262f4 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:25.962865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:26.134928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:26.171836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c48a1-0172-4ccc-b689-3eef47b9460a service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:26.183401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c48a1-0172-4ccc-b689-3eef47b9460a service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:26.198179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49902b17-0e94-4d3c-912c-40f9049f5fdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:26.199004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49902b17-0e94-4d3c-912c-40f9049f5fdf service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:26.199357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49902b17-0e94-4d3c-912c-40f9049f5fdf service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:26.199846 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49902b17-0e94-4d3c-912c-40f9049f5fdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:26.260648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b7fc275b-2120-4b40-8dbd-f4523b9262f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c83cc26f922442b68fea25edd3501248'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:26.315371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49902b17-0e94-4d3c-912c-40f9049f5fdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:26.318686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49902b17-0e94-4d3c-912c-40f9049f5fdf service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:26.333089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 841/1651] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:26.334056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49902b17-0e94-4d3c-912c-40f9049f5fdf service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:26.359407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b7fc275b-2120-4b40-8dbd-f4523b9262f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c83cc26f922442b68fea25edd3501248'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:26.361636 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b7fc275b-2120-4b40-8dbd-f4523b9262f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c83cc26f922442b68fea25edd3501248'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:26.363810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b7fc275b-2120-4b40-8dbd-f4523b9262f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c83cc26f922442b68fea25edd3501248'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:26.367045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49902b17-0e94-4d3c-912c-40f9049f5fdf service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:26.379446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b7fc275b-2120-4b40-8dbd-f4523b9262f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c83cc26f922442b68fea25edd3501248'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:26.387807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:26.388523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:26.389076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:26.394522 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] PATH_INFO: `/v3/users/0c8d72e1a3054b4db0a79e72a361c0be` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:26.395376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c8d72e1a3054b4db0a79e72a361c0be)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:26.399262 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: WARNING py.warnings [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:56:26.399262 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: warnings.warn(msg) Sep 09 12:56:26.399262 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: Sep 09 12:56:26.404513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:26.429849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 811/1652] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:25 2022] DELETE /identity/v3/users/c83cc26f922442b68fea25edd3501248 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:26.434282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee24c95-9823-4155-b83f-53b9fba52c90 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:26.434742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee24c95-9823-4155-b83f-53b9fba52c90 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:26.435094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee24c95-9823-4155-b83f-53b9fba52c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:26.486319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-5ee24c95-9823-4155-b83f-53b9fba52c90 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:26.489231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5ee24c95-9823-4155-b83f-53b9fba52c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:26.598636 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 812/1653] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:56:26.628049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee24c95-9823-4155-b83f-53b9fba52c90 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:26.631499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5ee24c95-9823-4155-b83f-53b9fba52c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:26.748706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee24c95-9823-4155-b83f-53b9fba52c90 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:26.753544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9050f-0d78-454f-b936-7d6017fc4acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:26.754536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9050f-0d78-454f-b936-7d6017fc4acf service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:26.754765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9050f-0d78-454f-b936-7d6017fc4acf service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:26.755102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9050f-0d78-454f-b936-7d6017fc4acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:26.767393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9050f-0d78-454f-b936-7d6017fc4acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:26.773237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9050f-0d78-454f-b936-7d6017fc4acf service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:26.775719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c8d72e1a3054b4db0a79e72a361c0be'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:26.790498 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 813/1654] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:26.796272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:26.859579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c8d72e1a3054b4db0a79e72a361c0be'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:26.861028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c8d72e1a3054b4db0a79e72a361c0be'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:26.862401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c8d72e1a3054b4db0a79e72a361c0be'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:26.909043 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c8d72e1a3054b4db0a79e72a361c0be'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:26.945339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 842/1655] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:26 2022] DELETE /identity/v3/users/0c8d72e1a3054b4db0a79e72a361c0be => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:26.950435 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34389-c8c6-4d72-b7c2-4f07a74ac9a2 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:26.951038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34389-c8c6-4d72-b7c2-4f07a74ac9a2 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:26.951289 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34389-c8c6-4d72-b7c2-4f07a74ac9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:26.986363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-b0a34389-c8c6-4d72-b7c2-4f07a74ac9a2 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:26.991657 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b0a34389-c8c6-4d72-b7c2-4f07a74ac9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:27.093178 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 814/1656] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:27.099388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:27.103710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:27.255298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a34389-c8c6-4d72-b7c2-4f07a74ac9a2 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:27.256908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b0a34389-c8c6-4d72-b7c2-4f07a74ac9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:27.306934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a34389-c8c6-4d72-b7c2-4f07a74ac9a2 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:27.331844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:27.334784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:27.337181 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:27.337181 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] PATH_INFO: `/v3/users/849c51110cfa47e0b2b5de048c6991ac` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:27.348499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=849c51110cfa47e0b2b5de048c6991ac)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:27.352022 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:27.384018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28114873-926f-436a-9b94-902cc43c130b admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:27.388758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b125b3f-b5e6-4a18-9100-f886866f1744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:27.390259 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b125b3f-b5e6-4a18-9100-f886866f1744 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:27.390674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b125b3f-b5e6-4a18-9100-f886866f1744 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:27.391643 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b125b3f-b5e6-4a18-9100-f886866f1744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:27.406752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b125b3f-b5e6-4a18-9100-f886866f1744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:27.408737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b125b3f-b5e6-4a18-9100-f886866f1744 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:27.467896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 843/1657] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:27.488071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:27.700890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:27.711625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '849c51110cfa47e0b2b5de048c6991ac'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:27.774395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '849c51110cfa47e0b2b5de048c6991ac'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:27.775096 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '849c51110cfa47e0b2b5de048c6991ac'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:27.775918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '849c51110cfa47e0b2b5de048c6991ac'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:27.793375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '849c51110cfa47e0b2b5de048c6991ac'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:27.823987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 815/1658] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:27 2022] DELETE /identity/v3/users/849c51110cfa47e0b2b5de048c6991ac => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:27.829030 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d214b-bf6d-4cfa-ba29-f2d03ac99a1a None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:27.829512 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d214b-bf6d-4cfa-ba29-f2d03ac99a1a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:27.829594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d214b-bf6d-4cfa-ba29-f2d03ac99a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:27.856145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b91d214b-bf6d-4cfa-ba29-f2d03ac99a1a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:27.858031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b91d214b-bf6d-4cfa-ba29-f2d03ac99a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:27.988028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 844/1659] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:27.988164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:27.989474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:28.057173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d214b-bf6d-4cfa-ba29-f2d03ac99a1a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:28.059227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b91d214b-bf6d-4cfa-ba29-f2d03ac99a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:28.088216 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d214b-bf6d-4cfa-ba29-f2d03ac99a1a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:28.093132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:28.094918 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:28.095205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:28.095481 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] PATH_INFO: `/v3/projects/fbb40c611f584fa6a4658e9159bc9351` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:28.096662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbb40c611f584fa6a4658e9159bc9351)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:28.100100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:28.161244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7a811-7d06-4868-a0d6-a3fc5b02c1c9 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:28.183108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700d837-b795-44f8-8707-377d6cd4c71e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:28.184060 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700d837-b795-44f8-8707-377d6cd4c71e service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:28.184060 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700d837-b795-44f8-8707-377d6cd4c71e service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:28.184313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700d837-b795-44f8-8707-377d6cd4c71e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:28.227467 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e700d837-b795-44f8-8707-377d6cd4c71e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:28.232712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e700d837-b795-44f8-8707-377d6cd4c71e service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:28.260209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 816/1660] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:28.289616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb40c611f584fa6a4658e9159bc9351', 'request_id': 'req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:28.308670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb40c611f584fa6a4658e9159bc9351', 'request_id': 'req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:28.415341 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb40c611f584fa6a4658e9159bc9351', 'request_id': 'req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:28.425722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] The token cache is being invalidate because project fbb40c611f584fa6a4658e9159bc9351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:28.426001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:28.426843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:28.428451 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 845/1661] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:28 2022] DELETE /identity/v3/projects/fbb40c611f584fa6a4658e9159bc9351 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:56:28.432736 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700d837-b795-44f8-8707-377d6cd4c71e service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:28.445160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700d837-b795-44f8-8707-377d6cd4c71e service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:28.591205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:28.592905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:28.632348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc056ec-8e1a-498b-a306-e41b17d96cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:28.633696 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc056ec-8e1a-498b-a306-e41b17d96cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:28.634042 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc056ec-8e1a-498b-a306-e41b17d96cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:28.634375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc056ec-8e1a-498b-a306-e41b17d96cdb admin admin] PATH_INFO: `/v3/users/68f4c80a7fb44dc4a70445f4ed6c7845` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:28.639148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc056ec-8e1a-498b-a306-e41b17d96cdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=68f4c80a7fb44dc4a70445f4ed6c7845)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:28.639148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc056ec-8e1a-498b-a306-e41b17d96cdb admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:28.756847 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17f5cb-7daa-4dfe-bab1-c8e09024ad71 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:28.769462 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095445dd-d728-4040-9814-13a0d7a885c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:28.770627 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095445dd-d728-4040-9814-13a0d7a885c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:28.771039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095445dd-d728-4040-9814-13a0d7a885c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:28.772618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095445dd-d728-4040-9814-13a0d7a885c8 admin admin] PATH_INFO: `/v3/projects/3a03f8b7f17247c18fa91a7d64a98eb3` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:28.774013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095445dd-d728-4040-9814-13a0d7a885c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a03f8b7f17247c18fa91a7d64a98eb3)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:28.776786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095445dd-d728-4040-9814-13a0d7a885c8 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:28.955594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-095445dd-d728-4040-9814-13a0d7a885c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a03f8b7f17247c18fa91a7d64a98eb3', 'request_id': 'req-095445dd-d728-4040-9814-13a0d7a885c8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:28.958508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5fc056ec-8e1a-498b-a306-e41b17d96cdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68f4c80a7fb44dc4a70445f4ed6c7845'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:28.967654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-095445dd-d728-4040-9814-13a0d7a885c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a03f8b7f17247c18fa91a7d64a98eb3', 'request_id': 'req-095445dd-d728-4040-9814-13a0d7a885c8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:29.031807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5fc056ec-8e1a-498b-a306-e41b17d96cdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68f4c80a7fb44dc4a70445f4ed6c7845'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:29.035833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5fc056ec-8e1a-498b-a306-e41b17d96cdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68f4c80a7fb44dc4a70445f4ed6c7845'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:29.035833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5fc056ec-8e1a-498b-a306-e41b17d96cdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68f4c80a7fb44dc4a70445f4ed6c7845'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:29.046656 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5fc056ec-8e1a-498b-a306-e41b17d96cdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68f4c80a7fb44dc4a70445f4ed6c7845'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:29.092602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 817/1662] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:28 2022] DELETE /identity/v3/users/68f4c80a7fb44dc4a70445f4ed6c7845 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:29.146252 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-095445dd-d728-4040-9814-13a0d7a885c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a03f8b7f17247c18fa91a7d64a98eb3', 'request_id': 'req-095445dd-d728-4040-9814-13a0d7a885c8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:29.167942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-095445dd-d728-4040-9814-13a0d7a885c8 admin admin] The token cache is being invalidate because project 3a03f8b7f17247c18fa91a7d64a98eb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:29.168340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-095445dd-d728-4040-9814-13a0d7a885c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:29.170152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-095445dd-d728-4040-9814-13a0d7a885c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:29.202609 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 846/1663] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:28 2022] DELETE /identity/v3/projects/3a03f8b7f17247c18fa91a7d64a98eb3 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:29.208533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c4303-7800-4abb-8db0-7b363160660e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:29.208925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c4303-7800-4abb-8db0-7b363160660e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:29.209203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c4303-7800-4abb-8db0-7b363160660e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:29.240385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-4a9c4303-7800-4abb-8db0-7b363160660e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:29.247012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-4a9c4303-7800-4abb-8db0-7b363160660e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:29.348236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 818/1664] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:29.356078 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e88611-129a-4c1d-98e5-d1b1bdf84cf3 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:29.357865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e88611-129a-4c1d-98e5-d1b1bdf84cf3 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:29.358308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e88611-129a-4c1d-98e5-d1b1bdf84cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:29.384618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-96e88611-129a-4c1d-98e5-d1b1bdf84cf3 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:29.386219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-96e88611-129a-4c1d-98e5-d1b1bdf84cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:29.408798 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 847/1665] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:29.413433 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c4303-7800-4abb-8db0-7b363160660e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:29.428022 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c4303-7800-4abb-8db0-7b363160660e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:29.434759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cce423-3317-4cd8-afc5-c795e90a64f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:29.437130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cce423-3317-4cd8-afc5-c795e90a64f4 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:29.437130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cce423-3317-4cd8-afc5-c795e90a64f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:29.437508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cce423-3317-4cd8-afc5-c795e90a64f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:29.438245 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cce423-3317-4cd8-afc5-c795e90a64f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:29.438697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cce423-3317-4cd8-afc5-c795e90a64f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:29.440538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cce423-3317-4cd8-afc5-c795e90a64f4 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:29.457183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 819/1666] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:29.461881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e88611-129a-4c1d-98e5-d1b1bdf84cf3 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:29.474647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e88611-129a-4c1d-98e5-d1b1bdf84cf3 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:29.480083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b0ca8-63c6-431b-8c21-86226e8ba7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:29.481023 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b0ca8-63c6-431b-8c21-86226e8ba7ff admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:29.481538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b0ca8-63c6-431b-8c21-86226e8ba7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:29.481996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b0ca8-63c6-431b-8c21-86226e8ba7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:29.483374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b0ca8-63c6-431b-8c21-86226e8ba7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:29.485711 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b0ca8-63c6-431b-8c21-86226e8ba7ff admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:29.589783 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 848/1667] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:29 2022] POST /identity/v3/projects => generated 403 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:29.594086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cce423-3317-4cd8-afc5-c795e90a64f4 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:29.603779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:29.609163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cce423-3317-4cd8-afc5-c795e90a64f4 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:29.746912 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca95c9-1f2c-4048-9846-2da1ff806fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:29.747606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca95c9-1f2c-4048-9846-2da1ff806fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:29.747885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca95c9-1f2c-4048-9846-2da1ff806fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:29.748155 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca95c9-1f2c-4048-9846-2da1ff806fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:29.748903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca95c9-1f2c-4048-9846-2da1ff806fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:29.751831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca95c9-1f2c-4048-9846-2da1ff806fa2 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:29.894622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 820/1668] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:29 2022] POST /identity/v3/users => generated 302 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:29.897192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b0ca8-63c6-431b-8c21-86226e8ba7ff admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:29.914144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b0ca8-63c6-431b-8c21-86226e8ba7ff admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:29.931237 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8a611-f480-4c51-980a-71a4987001e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:29.932303 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8a611-f480-4c51-980a-71a4987001e6 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:29.932752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8a611-f480-4c51-980a-71a4987001e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:29.933175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8a611-f480-4c51-980a-71a4987001e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:29.933981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8a611-f480-4c51-980a-71a4987001e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:29.935770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8a611-f480-4c51-980a-71a4987001e6 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:29.956123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 849/1669] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:29.960827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca95c9-1f2c-4048-9846-2da1ff806fa2 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:29.994764 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca95c9-1f2c-4048-9846-2da1ff806fa2 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:30.012296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f88243-961b-4959-a186-20802c79f209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:30.012296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f88243-961b-4959-a186-20802c79f209 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:30.012296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f88243-961b-4959-a186-20802c79f209 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:30.012296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f88243-961b-4959-a186-20802c79f209 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:30.012651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f88243-961b-4959-a186-20802c79f209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:30.013012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f88243-961b-4959-a186-20802c79f209 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:30.043905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 821/1670] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:30.046730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8a611-f480-4c51-980a-71a4987001e6 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:30.071135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8a611-f480-4c51-980a-71a4987001e6 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:30.085592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7e403-fb51-4119-9197-a8c69ab98cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:30.086356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7e403-fb51-4119-9197-a8c69ab98cfb admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:30.086667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7e403-fb51-4119-9197-a8c69ab98cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:30.086950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7e403-fb51-4119-9197-a8c69ab98cfb admin admin] PATH_INFO: `/v3/projects/7e331bbd70a44cad953fde0f67795e6f/users/39e24a359768468eb7600099b8020bd1/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:30.129357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7e403-fb51-4119-9197-a8c69ab98cfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e331bbd70a44cad953fde0f67795e6f, user_id=39e24a359768468eb7600099b8020bd1, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:30.136555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7e403-fb51-4119-9197-a8c69ab98cfb admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:30.164414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f88243-961b-4959-a186-20802c79f209 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:30.170837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e5f88243-961b-4959-a186-20802c79f209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:30.197846 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f88243-961b-4959-a186-20802c79f209 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:30.211144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:30.211805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:30.212090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:30.212430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] PATH_INFO: `/v3/projects/7df3b6aedfe14bd1b8263b75027f80b1` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:30.213658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df3b6aedfe14bd1b8263b75027f80b1)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:30.214717 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: WARNING py.warnings [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:56:30.214717 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: warnings.warn(msg) Sep 09 12:56:30.214717 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: Sep 09 12:56:30.215967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:30.225640 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:30.265682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 850/1671] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:30 2022] PUT /identity/v3/projects/7e331bbd70a44cad953fde0f67795e6f/users/39e24a359768468eb7600099b8020bd1/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:30.282862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b5b7e-b412-4969-a067-f9cb69f70145 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:30.283384 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b5b7e-b412-4969-a067-f9cb69f70145 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:30.283745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b5b7e-b412-4969-a067-f9cb69f70145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:30.371776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-636b5b7e-b412-4969-a067-f9cb69f70145 None None] MFA Rules not processed for user `39e24a359768468eb7600099b8020bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:30.375565 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-636b5b7e-b412-4969-a067-f9cb69f70145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:30.477863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df3b6aedfe14bd1b8263b75027f80b1', 'request_id': 'req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:30.538134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df3b6aedfe14bd1b8263b75027f80b1', 'request_id': 'req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:30.621406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df3b6aedfe14bd1b8263b75027f80b1', 'request_id': 'req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:30.621583 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 851/1672] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:30 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:30.626348 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b5b7e-b412-4969-a067-f9cb69f70145 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:30.644671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] The token cache is being invalidate because project 7df3b6aedfe14bd1b8263b75027f80b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:30.644904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:30.649777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:30.660142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 822/1673] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:30 2022] DELETE /identity/v3/projects/7df3b6aedfe14bd1b8263b75027f80b1 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:30.660991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:30.671669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:30.847292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b5b7e-b412-4969-a067-f9cb69f70145 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:30.870283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0af48d-c5c0-49d4-808f-8f2bf630cd7d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:30.875380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:30.881358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:30.881358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:30.881358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] PATH_INFO: `/v3/users/c57df58aea51481192560ce8ec324703` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:30.885797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c57df58aea51481192560ce8ec324703)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:30.887847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:30.967505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2b5d5-191d-46b0-b785-f84bd9161e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:30.968179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2b5d5-191d-46b0-b785-f84bd9161e97 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:30.968469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2b5d5-191d-46b0-b785-f84bd9161e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:30.968716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2b5d5-191d-46b0-b785-f84bd9161e97 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:30.969347 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d2b5d5-191d-46b0-b785-f84bd9161e97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:30.971340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d2b5d5-191d-46b0-b785-f84bd9161e97 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:31.123419 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 852/1674] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:30 2022] POST /identity/v3/projects => generated 405 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:31.127327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2b5d5-191d-46b0-b785-f84bd9161e97 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:31.129998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-37d2b5d5-191d-46b0-b785-f84bd9161e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:31.228296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c57df58aea51481192560ce8ec324703'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:31.238224 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:31.292609 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2b5d5-191d-46b0-b785-f84bd9161e97 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:31.297463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f3768-fc9b-47c0-a820-9d5562e2fa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:31.298598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f3768-fc9b-47c0-a820-9d5562e2fa91 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:31.298877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f3768-fc9b-47c0-a820-9d5562e2fa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:31.304156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f3768-fc9b-47c0-a820-9d5562e2fa91 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:31.304156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f3768-fc9b-47c0-a820-9d5562e2fa91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:31.304156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f3768-fc9b-47c0-a820-9d5562e2fa91 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:31.374519 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c57df58aea51481192560ce8ec324703'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:31.375201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c57df58aea51481192560ce8ec324703'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:31.375871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c57df58aea51481192560ce8ec324703'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:31.388504 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c57df58aea51481192560ce8ec324703'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:31.416758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 853/1675] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:31 2022] POST /identity/v3/users => generated 309 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:56:31.425514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f3768-fc9b-47c0-a820-9d5562e2fa91 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:31.430737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6a9f3768-fc9b-47c0-a820-9d5562e2fa91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:31.446371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 823/1676] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:30 2022] DELETE /identity/v3/users/c57df58aea51481192560ce8ec324703 => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:31.466760 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f3768-fc9b-47c0-a820-9d5562e2fa91 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:31.522654 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:31.525574 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:31.625839 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23038485-1c60-4e7b-ae4f-ecbad54ccf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:31.626401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23038485-1c60-4e7b-ae4f-ecbad54ccf88 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:31.626716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23038485-1c60-4e7b-ae4f-ecbad54ccf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:31.626994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23038485-1c60-4e7b-ae4f-ecbad54ccf88 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:31.627660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23038485-1c60-4e7b-ae4f-ecbad54ccf88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:31.631363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23038485-1c60-4e7b-ae4f-ecbad54ccf88 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:31.679776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 854/1677] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:31.680749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23038485-1c60-4e7b-ae4f-ecbad54ccf88 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:31.707476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23038485-1c60-4e7b-ae4f-ecbad54ccf88 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:31.708672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb132b96-9ec9-42b6-a711-d66d4ba698d3 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:31.714336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8598c-5700-4e95-855a-a611e4bca99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:31.715475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8598c-5700-4e95-855a-a611e4bca99e admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:31.715681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8598c-5700-4e95-855a-a611e4bca99e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:31.715966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8598c-5700-4e95-855a-a611e4bca99e admin admin] PATH_INFO: `/v3/projects/bef8d95c84244f19abac41398ffc88e9/users/4fd8d8362c774cdaaafc7c7ea15f8d84/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:31.721028 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:31.721838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:31.729389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:31.729790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b admin admin] PATH_INFO: `/v3/projects/1002db0c193e41629578594fa645df2d` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:31.731221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1002db0c193e41629578594fa645df2d)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:31.733321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:31.752182 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8598c-5700-4e95-855a-a611e4bca99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef8d95c84244f19abac41398ffc88e9, user_id=4fd8d8362c774cdaaafc7c7ea15f8d84, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:31.754834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8598c-5700-4e95-855a-a611e4bca99e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:31.780772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:31.868734 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 855/1678] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:31 2022] PUT /identity/v3/projects/bef8d95c84244f19abac41398ffc88e9/users/4fd8d8362c774cdaaafc7c7ea15f8d84/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:31.872944 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8598c-5700-4e95-855a-a611e4bca99e admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:31.921508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1002db0c193e41629578594fa645df2d', 'request_id': 'req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:31.949616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1002db0c193e41629578594fa645df2d', 'request_id': 'req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:32.037094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1002db0c193e41629578594fa645df2d', 'request_id': 'req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:32.058106 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b admin admin] The token cache is being invalidate because project 1002db0c193e41629578594fa645df2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:32.059242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:32.061154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6c3cc91a-bcb4-4835-8c7e-86bf36269f3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:32.065104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 824/1679] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:31 2022] DELETE /identity/v3/projects/1002db0c193e41629578594fa645df2d => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:32.079278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dc08b-b284-4d1c-bfa1-ee9d8c745585 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:32.079278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dc08b-b284-4d1c-bfa1-ee9d8c745585 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:32.079278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dc08b-b284-4d1c-bfa1-ee9d8c745585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:32.137789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ae6dc08b-b284-4d1c-bfa1-ee9d8c745585 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:32.139317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ae6dc08b-b284-4d1c-bfa1-ee9d8c745585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:32.151129 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8598c-5700-4e95-855a-a611e4bca99e admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:32.283161 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 825/1680] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:56:32.293049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f766d-d618-4cc8-9cdd-402c6bd5d728 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:32.293547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f766d-d618-4cc8-9cdd-402c6bd5d728 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:32.294081 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f766d-d618-4cc8-9cdd-402c6bd5d728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:32.301439 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab518d-7aa5-4776-8af2-ebce20f421fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:32.303712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab518d-7aa5-4776-8af2-ebce20f421fc admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:32.304113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab518d-7aa5-4776-8af2-ebce20f421fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:32.307361 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab518d-7aa5-4776-8af2-ebce20f421fc admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:32.307361 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab518d-7aa5-4776-8af2-ebce20f421fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:32.309322 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab518d-7aa5-4776-8af2-ebce20f421fc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:32.337776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-050f766d-d618-4cc8-9cdd-402c6bd5d728 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:32.339387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-050f766d-d618-4cc8-9cdd-402c6bd5d728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:32.349901 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 856/1681] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:32.353885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab518d-7aa5-4776-8af2-ebce20f421fc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:32.358785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b7ab518d-7aa5-4776-8af2-ebce20f421fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:32.364305 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 826/1682] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:56:32.369346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f766d-d618-4cc8-9cdd-402c6bd5d728 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:32.385745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f766d-d618-4cc8-9cdd-402c6bd5d728 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:32.389826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab518d-7aa5-4776-8af2-ebce20f421fc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:32.399338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439b0cc-ac50-44e3-8b7b-ce71bea8041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:32.400309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439b0cc-ac50-44e3-8b7b-ce71bea8041a admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:32.403002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439b0cc-ac50-44e3-8b7b-ce71bea8041a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:32.407314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439b0cc-ac50-44e3-8b7b-ce71bea8041a admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:32.407413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03a2cf-69d8-461b-9e07-3fa480fbd693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:32.407517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439b0cc-ac50-44e3-8b7b-ce71bea8041a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:32.407517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439b0cc-ac50-44e3-8b7b-ce71bea8041a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:32.407728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03a2cf-69d8-461b-9e07-3fa480fbd693 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:32.407728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03a2cf-69d8-461b-9e07-3fa480fbd693 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:32.407728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03a2cf-69d8-461b-9e07-3fa480fbd693 admin admin] PATH_INFO: `/v3/projects/bef8d95c84244f19abac41398ffc88e9/users/4fd8d8362c774cdaaafc7c7ea15f8d84/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:32.410709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439b0cc-ac50-44e3-8b7b-ce71bea8041a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:32.414042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03a2cf-69d8-461b-9e07-3fa480fbd693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef8d95c84244f19abac41398ffc88e9, user_id=4fd8d8362c774cdaaafc7c7ea15f8d84, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:32.416743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03a2cf-69d8-461b-9e07-3fa480fbd693 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:32.442245 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 827/1684] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:32.450166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 857/1684] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:32 2022] PUT /identity/v3/projects/bef8d95c84244f19abac41398ffc88e9/users/4fd8d8362c774cdaaafc7c7ea15f8d84/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:32.455909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03a2cf-69d8-461b-9e07-3fa480fbd693 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:32.456073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3776a-c53b-4d28-8f52-90f37b6f676f None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:32.459362 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3776a-c53b-4d28-8f52-90f37b6f676f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:32.459736 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3776a-c53b-4d28-8f52-90f37b6f676f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:32.526519 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-69e3776a-c53b-4d28-8f52-90f37b6f676f None None] MFA Rules not processed for user `4fd8d8362c774cdaaafc7c7ea15f8d84`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:32.528093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-69e3776a-c53b-4d28-8f52-90f37b6f676f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:32.563437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03a2cf-69d8-461b-9e07-3fa480fbd693 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:32.583720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622b4e-bb86-41e1-bd5e-375ab08ac61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:32.583720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622b4e-bb86-41e1-bd5e-375ab08ac61c admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:32.583720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622b4e-bb86-41e1-bd5e-375ab08ac61c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:32.583720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622b4e-bb86-41e1-bd5e-375ab08ac61c admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:32.586995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69622b4e-bb86-41e1-bd5e-375ab08ac61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:32.587145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69622b4e-bb86-41e1-bd5e-375ab08ac61c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:32.724727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 858/1685] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:32 2022] POST /identity/v3/projects => generated 375 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:32.724727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622b4e-bb86-41e1-bd5e-375ab08ac61c admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:32.727946 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-69622b4e-bb86-41e1-bd5e-375ab08ac61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:32.733268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 828/1686] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:32 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:32.746023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3776a-c53b-4d28-8f52-90f37b6f676f None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:32.758317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3776a-c53b-4d28-8f52-90f37b6f676f None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:32.871917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39537c-9b7a-4c0e-8758-de56db962e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:32.873263 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39537c-9b7a-4c0e-8758-de56db962e83 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:32.875672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39537c-9b7a-4c0e-8758-de56db962e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:32.875672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39537c-9b7a-4c0e-8758-de56db962e83 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:32.876456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39537c-9b7a-4c0e-8758-de56db962e83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:32.882717 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39537c-9b7a-4c0e-8758-de56db962e83 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:32.918715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622b4e-bb86-41e1-bd5e-375ab08ac61c admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:32.924794 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:32.926122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:32.927145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:32.927732 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] PATH_INFO: `/v3/projects/015d08a67b8a4d509263b36b2a4ffb06` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:32.929330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=015d08a67b8a4d509263b36b2a4ffb06)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:32.932021 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:33.045029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 829/1687] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:32 2022] POST /identity/v3/users => generated 288 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:33.047914 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39537c-9b7a-4c0e-8758-de56db962e83 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:33.053841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0f39537c-9b7a-4c0e-8758-de56db962e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:33.098370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '015d08a67b8a4d509263b36b2a4ffb06', 'request_id': 'req-4298dee3-372d-4542-809e-542d79ff0d01'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:33.115393 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '015d08a67b8a4d509263b36b2a4ffb06', 'request_id': 'req-4298dee3-372d-4542-809e-542d79ff0d01'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:33.215254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '015d08a67b8a4d509263b36b2a4ffb06', 'request_id': 'req-4298dee3-372d-4542-809e-542d79ff0d01'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:33.215804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39537c-9b7a-4c0e-8758-de56db962e83 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:33.232775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] The token cache is being invalidate because project 015d08a67b8a4d509263b36b2a4ffb06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:33.233078 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:33.234878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:33.236634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 859/1688] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:32 2022] DELETE /identity/v3/projects/015d08a67b8a4d509263b36b2a4ffb06 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:33.240893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:33.244430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:33.361324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0cd5d-0249-46d5-ab8f-b7ae63565e8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:33.362214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0cd5d-0249-46d5-ab8f-b7ae63565e8e service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:33.365935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0cd5d-0249-46d5-ab8f-b7ae63565e8e service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:33.365935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0cd5d-0249-46d5-ab8f-b7ae63565e8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:33.370905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b3f0cd5d-0249-46d5-ab8f-b7ae63565e8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:33.457735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298dee3-372d-4542-809e-542d79ff0d01 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:33.467957 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6bd38-8834-4c0d-a1b8-b772507da47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:33.470681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6bd38-8834-4c0d-a1b8-b772507da47b admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:33.474293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6bd38-8834-4c0d-a1b8-b772507da47b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:33.475012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6bd38-8834-4c0d-a1b8-b772507da47b admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:33.476670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6bd38-8834-4c0d-a1b8-b772507da47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:33.487807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6bd38-8834-4c0d-a1b8-b772507da47b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:33.511671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 860/1689] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:33.524539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5accf2-ebb5-4be5-aa45-ffc48eb583f8 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:33.524539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5accf2-ebb5-4be5-aa45-ffc48eb583f8 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:33.524539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5accf2-ebb5-4be5-aa45-ffc48eb583f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:33.538502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f0cd5d-0249-46d5-ab8f-b7ae63565e8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:33.541274 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f0cd5d-0249-46d5-ab8f-b7ae63565e8e service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:33.559650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-fd5accf2-ebb5-4be5-aa45-ffc48eb583f8 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:33.561313 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fd5accf2-ebb5-4be5-aa45-ffc48eb583f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:33.570608 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 830/1690] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:33 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:33.579828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 861/1691] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:33.580035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0cd5d-0249-46d5-ab8f-b7ae63565e8e service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:33.583434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964c689-9636-4c2c-9064-207063feac86 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:33.583723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964c689-9636-4c2c-9064-207063feac86 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:33.584002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964c689-9636-4c2c-9064-207063feac86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:33.609756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0cd5d-0249-46d5-ab8f-b7ae63565e8e service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:33.620120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f77c73-22f0-4f04-9276-a0233278aaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:33.620120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f77c73-22f0-4f04-9276-a0233278aaaa admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:33.620120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f77c73-22f0-4f04-9276-a0233278aaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:33.620120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f77c73-22f0-4f04-9276-a0233278aaaa admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:33.621146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f77c73-22f0-4f04-9276-a0233278aaaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:33.632547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f77c73-22f0-4f04-9276-a0233278aaaa admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:33.637786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:33.650308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d964c689-9636-4c2c-9064-207063feac86 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:33.652792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d964c689-9636-4c2c-9064-207063feac86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:33.665196 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 862/1692] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:33.671069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964c689-9636-4c2c-9064-207063feac86 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:33.683844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 831/1693] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:33.684839 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f77c73-22f0-4f04-9276-a0233278aaaa admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:33.702753 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964c689-9636-4c2c-9064-207063feac86 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:33.715086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f77c73-22f0-4f04-9276-a0233278aaaa admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:33.719721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df56690-fcff-4ff0-a65e-20cb09f03d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:33.720234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df56690-fcff-4ff0-a65e-20cb09f03d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:33.720547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df56690-fcff-4ff0-a65e-20cb09f03d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:33.720782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df56690-fcff-4ff0-a65e-20cb09f03d17 admin admin] PATH_INFO: `/v3/projects/199ddd039732406795849fb6044a4f72/users/b136a9a7f0324a34b6aa28ca59db6da3/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:33.735836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556c094-2eaa-41ff-95f5-6a50215b9516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:33.738147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556c094-2eaa-41ff-95f5-6a50215b9516 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:33.742905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556c094-2eaa-41ff-95f5-6a50215b9516 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:33.743439 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556c094-2eaa-41ff-95f5-6a50215b9516 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:33.745159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a556c094-2eaa-41ff-95f5-6a50215b9516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:33.747808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a556c094-2eaa-41ff-95f5-6a50215b9516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:33.751397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a556c094-2eaa-41ff-95f5-6a50215b9516 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:33.756025 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df56690-fcff-4ff0-a65e-20cb09f03d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=199ddd039732406795849fb6044a4f72, user_id=b136a9a7f0324a34b6aa28ca59db6da3, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:33.758786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df56690-fcff-4ff0-a65e-20cb09f03d17 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:33.796897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 863/1694] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:33.802455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556c094-2eaa-41ff-95f5-6a50215b9516 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:33.873649 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 832/1695] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:33 2022] PUT /identity/v3/projects/199ddd039732406795849fb6044a4f72/users/b136a9a7f0324a34b6aa28ca59db6da3/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:33.874219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556c094-2eaa-41ff-95f5-6a50215b9516 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:33.885817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01c470-4e29-47d6-aa8b-ae8a5f2eb183 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:33.890539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01c470-4e29-47d6-aa8b-ae8a5f2eb183 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:33.890978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01c470-4e29-47d6-aa8b-ae8a5f2eb183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:34.000240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-5a01c470-4e29-47d6-aa8b-ae8a5f2eb183 None None] MFA Rules not processed for user `b136a9a7f0324a34b6aa28ca59db6da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:34.002584 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5a01c470-4e29-47d6-aa8b-ae8a5f2eb183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:34.016263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:34.084562 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8dfe3-d7e0-44dc-987f-d3c8815121c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:34.085191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8dfe3-d7e0-44dc-987f-d3c8815121c3 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:34.085464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8dfe3-d7e0-44dc-987f-d3c8815121c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:34.087851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8dfe3-d7e0-44dc-987f-d3c8815121c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:34.088527 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8dfe3-d7e0-44dc-987f-d3c8815121c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:34.098320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8dfe3-d7e0-44dc-987f-d3c8815121c3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:34.164269 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 833/1696] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:56:34.165129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01c470-4e29-47d6-aa8b-ae8a5f2eb183 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:34.166706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5a01c470-4e29-47d6-aa8b-ae8a5f2eb183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:34.183482 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01c470-4e29-47d6-aa8b-ae8a5f2eb183 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:34.190291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e6ff2-2e62-4285-9e72-e79a9c6d670b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:34.191393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e6ff2-2e62-4285-9e72-e79a9c6d670b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:34.191719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e6ff2-2e62-4285-9e72-e79a9c6d670b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:34.191996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e6ff2-2e62-4285-9e72-e79a9c6d670b admin admin] PATH_INFO: `/v3/users/39e24a359768468eb7600099b8020bd1` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:34.194458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e6ff2-2e62-4285-9e72-e79a9c6d670b admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e24a359768468eb7600099b8020bd1)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:34.196538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e6ff2-2e62-4285-9e72-e79a9c6d670b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:34.235838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 864/1697] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:33 2022] POST /identity/v3/projects => generated 363 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:34.236646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8dfe3-d7e0-44dc-987f-d3c8815121c3 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:34.245901 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b8d8dfe3-d7e0-44dc-987f-d3c8815121c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:34.411261 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8dfe3-d7e0-44dc-987f-d3c8815121c3 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:34.417088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27020a1b-98be-42c1-85d4-32347917220d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff8fcb7fa2740bbb801badf800d2d6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:34.417919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27020a1b-98be-42c1-85d4-32347917220d service glance] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:34.418293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27020a1b-98be-42c1-85d4-32347917220d service glance] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:34.418699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27020a1b-98be-42c1-85d4-32347917220d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:34.504173 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1a3e6ff2-2e62-4285-9e72-e79a9c6d670b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e24a359768468eb7600099b8020bd1'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:34.527910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27020a1b-98be-42c1-85d4-32347917220d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:34.529729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27020a1b-98be-42c1-85d4-32347917220d service glance] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:34.542272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 865/1698] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:34.543275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27020a1b-98be-42c1-85d4-32347917220d service glance] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:34.637355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1a3e6ff2-2e62-4285-9e72-e79a9c6d670b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e24a359768468eb7600099b8020bd1'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:34.638205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1a3e6ff2-2e62-4285-9e72-e79a9c6d670b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e24a359768468eb7600099b8020bd1'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:34.638852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1a3e6ff2-2e62-4285-9e72-e79a9c6d670b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e24a359768468eb7600099b8020bd1'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:34.648920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1a3e6ff2-2e62-4285-9e72-e79a9c6d670b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e24a359768468eb7600099b8020bd1'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:34.725931 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27020a1b-98be-42c1-85d4-32347917220d service glance] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:34.750205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 834/1699] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:34 2022] DELETE /identity/v3/users/39e24a359768468eb7600099b8020bd1 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:34.754879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa41839-b6d8-451b-9372-9b8681c9f107 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:34.755171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa41839-b6d8-451b-9372-9b8681c9f107 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:34.755460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa41839-b6d8-451b-9372-9b8681c9f107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:34.761927 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c071b25-d8b0-4ca8-ae0f-ee1542eba446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:34.762616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c071b25-d8b0-4ca8-ae0f-ee1542eba446 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:34.762902 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c071b25-d8b0-4ca8-ae0f-ee1542eba446 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:34.763326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c071b25-d8b0-4ca8-ae0f-ee1542eba446 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:34.764651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c071b25-d8b0-4ca8-ae0f-ee1542eba446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:34.772551 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c071b25-d8b0-4ca8-ae0f-ee1542eba446 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:34.854176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-9aa41839-b6d8-451b-9372-9b8681c9f107 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:34.854176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9aa41839-b6d8-451b-9372-9b8681c9f107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:35.016891 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 866/1700] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:34 2022] POST /identity/v3/users => generated 282 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:35.017981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c071b25-d8b0-4ca8-ae0f-ee1542eba446 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:35.027317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4c071b25-d8b0-4ca8-ae0f-ee1542eba446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:35.064555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c071b25-d8b0-4ca8-ae0f-ee1542eba446 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:35.087226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a3220-a019-4ac6-ad20-328ba49a2fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:35.088137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a3220-a019-4ac6-ad20-328ba49a2fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:35.088739 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a3220-a019-4ac6-ad20-328ba49a2fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:35.089831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a3220-a019-4ac6-ad20-328ba49a2fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:35.094214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 835/1701] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:35.111087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a3220-a019-4ac6-ad20-328ba49a2fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:35.113178 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a3220-a019-4ac6-ad20-328ba49a2fc0 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:35.124873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa41839-b6d8-451b-9372-9b8681c9f107 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:35.126576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9aa41839-b6d8-451b-9372-9b8681c9f107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:35.161152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 867/1702] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:35.162146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a3220-a019-4ac6-ad20-328ba49a2fc0 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:35.181083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a3220-a019-4ac6-ad20-328ba49a2fc0 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:35.200095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffcb30-4532-4827-bf0b-4871165424e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:35.200722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffcb30-4532-4827-bf0b-4871165424e7 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:35.201033 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffcb30-4532-4827-bf0b-4871165424e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:35.201369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffcb30-4532-4827-bf0b-4871165424e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:35.202641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffcb30-4532-4827-bf0b-4871165424e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:35.204287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffcb30-4532-4827-bf0b-4871165424e7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:35.218230 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 868/1703] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:35.223323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffcb30-4532-4827-bf0b-4871165424e7 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:35.255970 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffcb30-4532-4827-bf0b-4871165424e7 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:35.277559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da816333-bfa9-4bb3-a167-6507e9852d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:35.278242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da816333-bfa9-4bb3-a167-6507e9852d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:35.278744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da816333-bfa9-4bb3-a167-6507e9852d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:35.279198 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da816333-bfa9-4bb3-a167-6507e9852d4f admin admin] PATH_INFO: `/v3/projects/e05ad8bb6540472b9eac5e3cf8c4fa45/users/274c5b2c58934a3eb3a9226d13bf01d4/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:35.317891 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da816333-bfa9-4bb3-a167-6507e9852d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e05ad8bb6540472b9eac5e3cf8c4fa45, user_id=274c5b2c58934a3eb3a9226d13bf01d4, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:35.320943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da816333-bfa9-4bb3-a167-6507e9852d4f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:35.329968 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa41839-b6d8-451b-9372-9b8681c9f107 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:35.346155 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735378b6-a928-45b6-8242-6497c838ef53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:35.351233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735378b6-a928-45b6-8242-6497c838ef53 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:35.351233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735378b6-a928-45b6-8242-6497c838ef53 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:35.351233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735378b6-a928-45b6-8242-6497c838ef53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:35.379697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735378b6-a928-45b6-8242-6497c838ef53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:35.381760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735378b6-a928-45b6-8242-6497c838ef53 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:35.398567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 836/1704] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:35.440629 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 869/1705] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:35 2022] PUT /identity/v3/projects/e05ad8bb6540472b9eac5e3cf8c4fa45/users/274c5b2c58934a3eb3a9226d13bf01d4/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:56:35.451964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05be49b-6bb6-4421-8a75-53dfdaa99c82 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:35.452732 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05be49b-6bb6-4421-8a75-53dfdaa99c82 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:35.453180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05be49b-6bb6-4421-8a75-53dfdaa99c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:35.507133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-f05be49b-6bb6-4421-8a75-53dfdaa99c82 None None] MFA Rules not processed for user `274c5b2c58934a3eb3a9226d13bf01d4`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:35.512882 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f05be49b-6bb6-4421-8a75-53dfdaa99c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:35.658530 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 837/1706] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:35.664792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da816333-bfa9-4bb3-a167-6507e9852d4f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:35.711715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:35.735900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05be49b-6bb6-4421-8a75-53dfdaa99c82 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:35.738399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f05be49b-6bb6-4421-8a75-53dfdaa99c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:35.831765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da816333-bfa9-4bb3-a167-6507e9852d4f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:35.853894 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc329ec-0f4f-4d08-b519-10f15862de65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:35.854549 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc329ec-0f4f-4d08-b519-10f15862de65 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:35.855171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc329ec-0f4f-4d08-b519-10f15862de65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:35.855171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc329ec-0f4f-4d08-b519-10f15862de65 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:35.855964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc329ec-0f4f-4d08-b519-10f15862de65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:35.860207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc329ec-0f4f-4d08-b519-10f15862de65 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:35.916012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05be49b-6bb6-4421-8a75-53dfdaa99c82 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:35.921424 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14cda9-a215-41f7-8dff-6310200ad667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:35.922372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14cda9-a215-41f7-8dff-6310200ad667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:35.922837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14cda9-a215-41f7-8dff-6310200ad667 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:35.923282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14cda9-a215-41f7-8dff-6310200ad667 admin admin] PATH_INFO: `/v3/users/6445de29c36d4f8f8ef51d2277e287ef` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:35.924609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14cda9-a215-41f7-8dff-6310200ad667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6445de29c36d4f8f8ef51d2277e287ef)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:35.927180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14cda9-a215-41f7-8dff-6310200ad667 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:36.055515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 870/1707] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:35 2022] POST /identity/v3/projects => generated 365 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:36.057815 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8515df-5272-4002-b891-dca6d31e8fb6 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:36.059189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8515df-5272-4002-b891-dca6d31e8fb6 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:36.059630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8515df-5272-4002-b891-dca6d31e8fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:36.127128 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-2c8515df-5272-4002-b891-dca6d31e8fb6 None None] MFA Rules not processed for user `c71eb53fc47c4f82add6fdd3d469230e`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:36.128441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2c8515df-5272-4002-b891-dca6d31e8fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:36.261029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 871/1708] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:36.263934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8515df-5272-4002-b891-dca6d31e8fb6 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:36.366766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-7f14cda9-a215-41f7-8dff-6310200ad667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6445de29c36d4f8f8ef51d2277e287ef'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:36.458069 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-7f14cda9-a215-41f7-8dff-6310200ad667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6445de29c36d4f8f8ef51d2277e287ef'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:36.460479 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-7f14cda9-a215-41f7-8dff-6310200ad667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6445de29c36d4f8f8ef51d2277e287ef'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:36.461377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-7f14cda9-a215-41f7-8dff-6310200ad667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6445de29c36d4f8f8ef51d2277e287ef'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:36.496874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-7f14cda9-a215-41f7-8dff-6310200ad667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6445de29c36d4f8f8ef51d2277e287ef'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:36.517031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:36.539244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8515df-5272-4002-b891-dca6d31e8fb6 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:36.544140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 838/1709] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:35 2022] DELETE /identity/v3/users/6445de29c36d4f8f8ef51d2277e287ef => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:36.551937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9292c-1716-45ce-94bf-5f844b3492df None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:36.552474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9292c-1716-45ce-94bf-5f844b3492df None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:36.554087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9292c-1716-45ce-94bf-5f844b3492df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:36.613602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-4ba9292c-1716-45ce-94bf-5f844b3492df None None] MFA Rules not processed for user `c71eb53fc47c4f82add6fdd3d469230e`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:36.615113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-4ba9292c-1716-45ce-94bf-5f844b3492df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:36.694918 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02f84b-fd10-4387-ae18-938a11014719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:36.695659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02f84b-fd10-4387-ae18-938a11014719 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:36.695947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02f84b-fd10-4387-ae18-938a11014719 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:36.696222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02f84b-fd10-4387-ae18-938a11014719 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:36.700125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c02f84b-fd10-4387-ae18-938a11014719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:36.707230 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c02f84b-fd10-4387-ae18-938a11014719 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:36.776707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 839/1710] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:36.778906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bd9d7-cc88-4a0d-aef5-d00db8680327 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:36.793082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bd9d7-cc88-4a0d-aef5-d00db8680327 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:36.793479 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bd9d7-cc88-4a0d-aef5-d00db8680327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:36.830002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-282bd9d7-cc88-4a0d-aef5-d00db8680327 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:36.831873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-282bd9d7-cc88-4a0d-aef5-d00db8680327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:36.835804 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 872/1711] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:36 2022] POST /identity/v3/users => generated 289 bytes in 570 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:36.836949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02f84b-fd10-4387-ae18-938a11014719 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:36.838907 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5c02f84b-fd10-4387-ae18-938a11014719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:36.843369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 840/1712] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:36.844414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bd9d7-cc88-4a0d-aef5-d00db8680327 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:36.845989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-282bd9d7-cc88-4a0d-aef5-d00db8680327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:36.869376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bd9d7-cc88-4a0d-aef5-d00db8680327 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:36.879582 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d9b18-a6ee-4ba2-bda2-5e0baa9ea329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:36.883624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d9b18-a6ee-4ba2-bda2-5e0baa9ea329 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:36.883624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d9b18-a6ee-4ba2-bda2-5e0baa9ea329 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:36.883624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d9b18-a6ee-4ba2-bda2-5e0baa9ea329 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:36.883624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d9b18-a6ee-4ba2-bda2-5e0baa9ea329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:36.884561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d9b18-a6ee-4ba2-bda2-5e0baa9ea329 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:36.899802 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 841/1713] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:36.903731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308cfb7-159f-4f25-b4e8-353993bdac05 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:36.906765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308cfb7-159f-4f25-b4e8-353993bdac05 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:36.906765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308cfb7-159f-4f25-b4e8-353993bdac05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:36.958424 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-e308cfb7-159f-4f25-b4e8-353993bdac05 None None] MFA Rules not processed for user `63a2902f465f4fd1aed80060c2669259`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:36.958989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e308cfb7-159f-4f25-b4e8-353993bdac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:37.106687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02f84b-fd10-4387-ae18-938a11014719 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:37.118814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2e663-d05e-432f-b998-d05b260bd917 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:37.120032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2e663-d05e-432f-b998-d05b260bd917 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:37.120290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2e663-d05e-432f-b998-d05b260bd917 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:37.120719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2e663-d05e-432f-b998-d05b260bd917 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:37.131041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1bc2e663-d05e-432f-b998-d05b260bd917 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:37.179687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2e663-d05e-432f-b998-d05b260bd917 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:37.186590 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2e663-d05e-432f-b998-d05b260bd917 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:37.204695 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 842/1714] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:37.209206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308cfb7-159f-4f25-b4e8-353993bdac05 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:37.212774 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e308cfb7-159f-4f25-b4e8-353993bdac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:37.226568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 873/1715] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:37.235671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2e663-d05e-432f-b998-d05b260bd917 service nova] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:37.279699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2e663-d05e-432f-b998-d05b260bd917 service nova] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:37.287003 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a911666-75a9-44a4-82c5-715c9544034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:37.288479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a911666-75a9-44a4-82c5-715c9544034c admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:37.288479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a911666-75a9-44a4-82c5-715c9544034c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:37.288479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a911666-75a9-44a4-82c5-715c9544034c admin admin] PATH_INFO: `/v3/projects/6c238134c7ac460da94da62a78c406d7/users/77f8b9fbd65d46c596fe2d430e171f65/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:37.326381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a911666-75a9-44a4-82c5-715c9544034c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c238134c7ac460da94da62a78c406d7, user_id=77f8b9fbd65d46c596fe2d430e171f65, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:37.330559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a911666-75a9-44a4-82c5-715c9544034c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:37.351737 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308cfb7-159f-4f25-b4e8-353993bdac05 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:37.355631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e100e3-e0ed-4094-871a-44f6e33314a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:37.356303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e100e3-e0ed-4094-871a-44f6e33314a3 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:37.356585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e100e3-e0ed-4094-871a-44f6e33314a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:37.356903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e100e3-e0ed-4094-871a-44f6e33314a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:37.369153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e100e3-e0ed-4094-871a-44f6e33314a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:37.370676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e100e3-e0ed-4094-871a-44f6e33314a3 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:37.389766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 843/1716] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:37.403621 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:37.405967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e100e3-e0ed-4094-871a-44f6e33314a3 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:37.408733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-78e100e3-e0ed-4094-871a-44f6e33314a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:37.459335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 874/1717] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:37 2022] PUT /identity/v3/projects/6c238134c7ac460da94da62a78c406d7/users/77f8b9fbd65d46c596fe2d430e171f65/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:37.462232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b04543-dd43-401b-8fd6-66717387f817 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:37.462528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b04543-dd43-401b-8fd6-66717387f817 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:37.463735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b04543-dd43-401b-8fd6-66717387f817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:37.464494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e100e3-e0ed-4094-871a-44f6e33314a3 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:37.587821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-14b04543-dd43-401b-8fd6-66717387f817 None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:37.589766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89212ffc-dc4d-4eed-a85a-9eea1362dd5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:37.590346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-14b04543-dd43-401b-8fd6-66717387f817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:37.590837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89212ffc-dc4d-4eed-a85a-9eea1362dd5f service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:37.591776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89212ffc-dc4d-4eed-a85a-9eea1362dd5f service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:37.593150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89212ffc-dc4d-4eed-a85a-9eea1362dd5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:37.686068 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89212ffc-dc4d-4eed-a85a-9eea1362dd5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:37.689229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89212ffc-dc4d-4eed-a85a-9eea1362dd5f service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:37.693006 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 875/1718] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 12:56:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:37.694472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b04543-dd43-401b-8fd6-66717387f817 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:37.707065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 844/1719] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:37.821071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89212ffc-dc4d-4eed-a85a-9eea1362dd5f service nova] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:37.823723 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-89212ffc-dc4d-4eed-a85a-9eea1362dd5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:37.950598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b04543-dd43-401b-8fd6-66717387f817 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:37.962108 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa119efa-5db9-43c4-9049-f8ebc5280dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:37.963968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa119efa-5db9-43c4-9049-f8ebc5280dff admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:37.964249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa119efa-5db9-43c4-9049-f8ebc5280dff admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:37.964520 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa119efa-5db9-43c4-9049-f8ebc5280dff admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:37.966740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa119efa-5db9-43c4-9049-f8ebc5280dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:37.970126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa119efa-5db9-43c4-9049-f8ebc5280dff admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:38.034227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 876/1720] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:38.040766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa119efa-5db9-43c4-9049-f8ebc5280dff admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:38.059804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89212ffc-dc4d-4eed-a85a-9eea1362dd5f service nova] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:38.064229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa119efa-5db9-43c4-9049-f8ebc5280dff admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:38.078122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:38.083362 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:38.084640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:38.085512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] PATH_INFO: `/v3/projects/7e331bbd70a44cad953fde0f67795e6f` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:38.090885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e331bbd70a44cad953fde0f67795e6f)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:38.094031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520bb25-f3e8-4a9a-8041-5afaa30c0af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:38.095903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:38.108449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520bb25-f3e8-4a9a-8041-5afaa30c0af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:38.117141 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520bb25-f3e8-4a9a-8041-5afaa30c0af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:38.124956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520bb25-f3e8-4a9a-8041-5afaa30c0af4 admin admin] PATH_INFO: `/v3/projects/6c238134c7ac460da94da62a78c406d7/users/77f8b9fbd65d46c596fe2d430e171f65/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:38.124956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520bb25-f3e8-4a9a-8041-5afaa30c0af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c238134c7ac460da94da62a78c406d7, user_id=77f8b9fbd65d46c596fe2d430e171f65, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:38.134709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520bb25-f3e8-4a9a-8041-5afaa30c0af4 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:38.177042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 877/1721] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:38 2022] PUT /identity/v3/projects/6c238134c7ac460da94da62a78c406d7/users/77f8b9fbd65d46c596fe2d430e171f65/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:38.178821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776561c3-ed05-4d6d-849d-698ea340a54c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:38.181949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776561c3-ed05-4d6d-849d-698ea340a54c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:38.181949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776561c3-ed05-4d6d-849d-698ea340a54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:38.248489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-776561c3-ed05-4d6d-849d-698ea340a54c None None] MFA Rules not processed for user `6ec951604cde4768beb02906caf6d7c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:38.249740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-776561c3-ed05-4d6d-849d-698ea340a54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:38.337441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e331bbd70a44cad953fde0f67795e6f', 'request_id': 'req-e5969be5-1cc6-48be-adca-9cc09281a1b3'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:38.347883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e331bbd70a44cad953fde0f67795e6f', 'request_id': 'req-e5969be5-1cc6-48be-adca-9cc09281a1b3'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:38.439416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 878/1722] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:38.439416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb0efb-700d-4375-9d9a-7a2a1e7adbc9 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:38.439416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb0efb-700d-4375-9d9a-7a2a1e7adbc9 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:38.439416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb0efb-700d-4375-9d9a-7a2a1e7adbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:38.465631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e331bbd70a44cad953fde0f67795e6f', 'request_id': 'req-e5969be5-1cc6-48be-adca-9cc09281a1b3'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:38.482135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] The token cache is being invalidate because project 7e331bbd70a44cad953fde0f67795e6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:38.482793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:38.488220 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-edbb0efb-700d-4375-9d9a-7a2a1e7adbc9 None None] MFA Rules not processed for user `77f8b9fbd65d46c596fe2d430e171f65`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:38.489804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:38.490230 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-edbb0efb-700d-4375-9d9a-7a2a1e7adbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:38.491824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 845/1723] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:37 2022] DELETE /identity/v3/projects/7e331bbd70a44cad953fde0f67795e6f => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:38.492579 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:38.494023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:38.689503 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 879/1724] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:38.691856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb0efb-700d-4375-9d9a-7a2a1e7adbc9 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:38.695687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-edbb0efb-700d-4375-9d9a-7a2a1e7adbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:38.697038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:38.733889 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5969be5-1cc6-48be-adca-9cc09281a1b3 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:38.749750 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873ba3b-ebf6-4488-85e2-06d65685edf7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:38.750391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873ba3b-ebf6-4488-85e2-06d65685edf7 service swift] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:38.750725 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873ba3b-ebf6-4488-85e2-06d65685edf7 service swift] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:38.750994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873ba3b-ebf6-4488-85e2-06d65685edf7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:38.753002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9873ba3b-ebf6-4488-85e2-06d65685edf7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:38.846066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb0efb-700d-4375-9d9a-7a2a1e7adbc9 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:38.869265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:38.869929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:38.870212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:38.870518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] PATH_INFO: `/v3/projects/99de43adb16f490fa1ab22985b27c1c3` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:38.871686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99de43adb16f490fa1ab22985b27c1c3)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:38.873805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:38.916535 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873ba3b-ebf6-4488-85e2-06d65685edf7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:38.918170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873ba3b-ebf6-4488-85e2-06d65685edf7 service swift] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:38.930183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 846/1725] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 12:56:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:56:38.932193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a911776-9638-4e04-92d9-1d838a226b80 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:38.932193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a911776-9638-4e04-92d9-1d838a226b80 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:38.932193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a911776-9638-4e04-92d9-1d838a226b80 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:38.933592 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 847/1726] 10.222.0.46 () {54 vars in 912 bytes} [Fri Sep 9 12:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:56:38.934306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a911776-9638-4e04-92d9-1d838a226b80 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:38.936024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9a911776-9638-4e04-92d9-1d838a226b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:39.118629 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99de43adb16f490fa1ab22985b27c1c3', 'request_id': 'req-03620781-5edf-42b6-b94c-c6fc6aaaffa9'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:39.133898 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99de43adb16f490fa1ab22985b27c1c3', 'request_id': 'req-03620781-5edf-42b6-b94c-c6fc6aaaffa9'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:39.175474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a911776-9638-4e04-92d9-1d838a226b80 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:39.261401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99de43adb16f490fa1ab22985b27c1c3', 'request_id': 'req-03620781-5edf-42b6-b94c-c6fc6aaaffa9'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:39.288522 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] The token cache is being invalidate because project 99de43adb16f490fa1ab22985b27c1c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:39.288828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:39.295147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:39.297042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 880/1727] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:38 2022] DELETE /identity/v3/projects/99de43adb16f490fa1ab22985b27c1c3 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:39.304199 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:39.313899 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:39.427271 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4203ee-c636-43c4-8e7e-1388e0afc811 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:39.430255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4203ee-c636-43c4-8e7e-1388e0afc811 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:39.430255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4203ee-c636-43c4-8e7e-1388e0afc811 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:39.430255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4203ee-c636-43c4-8e7e-1388e0afc811 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:39.437975 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9f4203ee-c636-43c4-8e7e-1388e0afc811 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:39.552922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03620781-5edf-42b6-b94c-c6fc6aaaffa9 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:39.559897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cc82f-817b-40ce-b2fc-87a9636de4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:39.562201 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cc82f-817b-40ce-b2fc-87a9636de4fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:39.562201 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cc82f-817b-40ce-b2fc-87a9636de4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:39.562201 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cc82f-817b-40ce-b2fc-87a9636de4fd admin admin] PATH_INFO: `/v3/users/4fd8d8362c774cdaaafc7c7ea15f8d84` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:39.563910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cc82f-817b-40ce-b2fc-87a9636de4fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fd8d8362c774cdaaafc7c7ea15f8d84)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:39.568352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cc82f-817b-40ce-b2fc-87a9636de4fd admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:39.578779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4203ee-c636-43c4-8e7e-1388e0afc811 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:39.582358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4203ee-c636-43c4-8e7e-1388e0afc811 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:39.611011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 848/1728] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:39.612488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4203ee-c636-43c4-8e7e-1388e0afc811 service nova] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:39.619550 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9f4203ee-c636-43c4-8e7e-1388e0afc811 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:39.777197 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4203ee-c636-43c4-8e7e-1388e0afc811 service nova] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:39.790456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba540a6-ab41-416e-8c26-8dbaefb9c37b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:39.793330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba540a6-ab41-416e-8c26-8dbaefb9c37b service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:39.797057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba540a6-ab41-416e-8c26-8dbaefb9c37b service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:39.797711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba540a6-ab41-416e-8c26-8dbaefb9c37b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:39.801847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2ba540a6-ab41-416e-8c26-8dbaefb9c37b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:39.816478 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:39.822850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-334cc82f-817b-40ce-b2fc-87a9636de4fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fd8d8362c774cdaaafc7c7ea15f8d84'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:39.942133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba540a6-ab41-416e-8c26-8dbaefb9c37b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:39.948054 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba540a6-ab41-416e-8c26-8dbaefb9c37b service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:39.962849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-334cc82f-817b-40ce-b2fc-87a9636de4fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fd8d8362c774cdaaafc7c7ea15f8d84'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:39.965219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-334cc82f-817b-40ce-b2fc-87a9636de4fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fd8d8362c774cdaaafc7c7ea15f8d84'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:39.965219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-334cc82f-817b-40ce-b2fc-87a9636de4fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fd8d8362c774cdaaafc7c7ea15f8d84'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:39.993139 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-334cc82f-817b-40ce-b2fc-87a9636de4fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fd8d8362c774cdaaafc7c7ea15f8d84'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:40.007427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 849/1729] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:40.007427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdbabc-9857-4ce6-8cf6-92fe53541427 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:40.007427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdbabc-9857-4ce6-8cf6-92fe53541427 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:40.007427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdbabc-9857-4ce6-8cf6-92fe53541427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:40.022985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 881/1730] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:39 2022] DELETE /identity/v3/users/4fd8d8362c774cdaaafc7c7ea15f8d84 => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:40.025168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd4f17-5470-4aaa-b050-65c4fadaaf9c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:40.025627 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd4f17-5470-4aaa-b050-65c4fadaaf9c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:40.026106 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd4f17-5470-4aaa-b050-65c4fadaaf9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:40.084904 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-fffd4f17-5470-4aaa-b050-65c4fadaaf9c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:40.088349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fffd4f17-5470-4aaa-b050-65c4fadaaf9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:40.093158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-16cdbabc-9857-4ce6-8cf6-92fe53541427 None None] MFA Rules not processed for user `e46bd9d49bf64013ae58770b0284d81e`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:40.103640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-16cdbabc-9857-4ce6-8cf6-92fe53541427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:40.200245 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 850/1731] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 12:56:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:40.215912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 882/1732] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:40.224020 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e5856-eb07-40d8-a5e1-3d6bfd2deee7 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:40.224306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e5856-eb07-40d8-a5e1-3d6bfd2deee7 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:40.224580 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e5856-eb07-40d8-a5e1-3d6bfd2deee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:40.255882 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-a22e5856-eb07-40d8-a5e1-3d6bfd2deee7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:40.257451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a22e5856-eb07-40d8-a5e1-3d6bfd2deee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:40.260478 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd4f17-5470-4aaa-b050-65c4fadaaf9c None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:40.262263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fffd4f17-5470-4aaa-b050-65c4fadaaf9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:40.301709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 851/1733] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:40.302892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e5856-eb07-40d8-a5e1-3d6bfd2deee7 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:40.306271 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a22e5856-eb07-40d8-a5e1-3d6bfd2deee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:40.364258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e5856-eb07-40d8-a5e1-3d6bfd2deee7 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:40.374292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd5e7d-b7a1-4361-87f7-331e0e6f06e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:40.375091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd5e7d-b7a1-4361-87f7-331e0e6f06e2 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:40.376034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd5e7d-b7a1-4361-87f7-331e0e6f06e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:40.376034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd5e7d-b7a1-4361-87f7-331e0e6f06e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:40.388372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd4f17-5470-4aaa-b050-65c4fadaaf9c None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:40.393343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543a473-3e56-4881-8d6e-039a4e91548d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:40.394676 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543a473-3e56-4881-8d6e-039a4e91548d service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:40.395040 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543a473-3e56-4881-8d6e-039a4e91548d service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:40.395201 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543a473-3e56-4881-8d6e-039a4e91548d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:40.395553 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd5e7d-b7a1-4361-87f7-331e0e6f06e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:40.397166 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd5e7d-b7a1-4361-87f7-331e0e6f06e2 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:40.397577 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d543a473-3e56-4881-8d6e-039a4e91548d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:40.423570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 852/1734] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:40.424610 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd5e7d-b7a1-4361-87f7-331e0e6f06e2 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:40.459497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd5e7d-b7a1-4361-87f7-331e0e6f06e2 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:40.470995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af2ee6-11bb-415b-962c-3a36b50f900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:40.472372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af2ee6-11bb-415b-962c-3a36b50f900e admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:40.472750 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af2ee6-11bb-415b-962c-3a36b50f900e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:40.473117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af2ee6-11bb-415b-962c-3a36b50f900e admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:40.473866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af2ee6-11bb-415b-962c-3a36b50f900e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:40.474366 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af2ee6-11bb-415b-962c-3a36b50f900e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:40.476342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af2ee6-11bb-415b-962c-3a36b50f900e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:40.494031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 853/1735] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:40.499129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af2ee6-11bb-415b-962c-3a36b50f900e admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:40.548307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af2ee6-11bb-415b-962c-3a36b50f900e admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:40.566511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59e6b4-4266-494d-b682-184fd54ca3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:40.567381 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59e6b4-4266-494d-b682-184fd54ca3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:40.567779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59e6b4-4266-494d-b682-184fd54ca3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:40.568157 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59e6b4-4266-494d-b682-184fd54ca3f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:40.569028 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59e6b4-4266-494d-b682-184fd54ca3f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:40.569649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543a473-3e56-4881-8d6e-039a4e91548d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:40.571436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59e6b4-4266-494d-b682-184fd54ca3f9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:40.572023 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543a473-3e56-4881-8d6e-039a4e91548d service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:40.584706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 883/1736] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:56:40.732674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 854/1737] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:40 2022] POST /identity/v3/projects => generated 363 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:40.736680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543a473-3e56-4881-8d6e-039a4e91548d service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:40.757520 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543a473-3e56-4881-8d6e-039a4e91548d service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:40.826190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1b380-1dcf-4615-b542-3f41d67c057c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:40.828154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1b380-1dcf-4615-b542-3f41d67c057c admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:40.828154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1b380-1dcf-4615-b542-3f41d67c057c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:40.828154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1b380-1dcf-4615-b542-3f41d67c057c admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:40.829361 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1b380-1dcf-4615-b542-3f41d67c057c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:40.833686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1b380-1dcf-4615-b542-3f41d67c057c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:40.863036 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59e6b4-4266-494d-b682-184fd54ca3f9 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:40.864778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0e59e6b4-4266-494d-b682-184fd54ca3f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:40.902920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59e6b4-4266-494d-b682-184fd54ca3f9 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:40.919244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:40.919858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:40.920423 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:40.920742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8 admin admin] PATH_INFO: `/v3/projects/bef8d95c84244f19abac41398ffc88e9` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:40.921744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 884/1738] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:40 2022] POST /identity/v3/users => generated 282 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:56:40.922302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bef8d95c84244f19abac41398ffc88e9)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:40.922937 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1b380-1dcf-4615-b542-3f41d67c057c admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:40.925501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:40.948675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1b380-1dcf-4615-b542-3f41d67c057c admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:40.965955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56371006-c6e3-4a21-9c8a-57617631a648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:40.966722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56371006-c6e3-4a21-9c8a-57617631a648 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:40.967133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56371006-c6e3-4a21-9c8a-57617631a648 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:40.967502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56371006-c6e3-4a21-9c8a-57617631a648 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:40.968264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56371006-c6e3-4a21-9c8a-57617631a648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:40.970397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56371006-c6e3-4a21-9c8a-57617631a648 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:41.006726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 885/1739] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:41.008759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56371006-c6e3-4a21-9c8a-57617631a648 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:41.036079 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56371006-c6e3-4a21-9c8a-57617631a648 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:41.054843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed8fe5-ed9c-4d79-bf70-3d9758dbb02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:41.054843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed8fe5-ed9c-4d79-bf70-3d9758dbb02d admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:41.054843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed8fe5-ed9c-4d79-bf70-3d9758dbb02d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:41.054843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed8fe5-ed9c-4d79-bf70-3d9758dbb02d admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:41.057185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed8fe5-ed9c-4d79-bf70-3d9758dbb02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:41.057783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:41.074526 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed8fe5-ed9c-4d79-bf70-3d9758dbb02d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:41.091470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 886/1740] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:56:41.097108 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed8fe5-ed9c-4d79-bf70-3d9758dbb02d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:41.131478 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed8fe5-ed9c-4d79-bf70-3d9758dbb02d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:41.241537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:41.245106 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef8d95c84244f19abac41398ffc88e9', 'request_id': 'req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:41.274725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce4396-42f7-4a3a-b890-07356d9e86fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:41.276047 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce4396-42f7-4a3a-b890-07356d9e86fd admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:41.277582 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce4396-42f7-4a3a-b890-07356d9e86fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:41.278981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce4396-42f7-4a3a-b890-07356d9e86fd admin admin] PATH_INFO: `/v3/projects/c55500b03ab041979cd6d7e3a8bde147/users/b18cff690f7e408da812ce7f02f4b2d5/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:41.281287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef8d95c84244f19abac41398ffc88e9', 'request_id': 'req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:41.313982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce4396-42f7-4a3a-b890-07356d9e86fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c55500b03ab041979cd6d7e3a8bde147, user_id=b18cff690f7e408da812ce7f02f4b2d5, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:41.322544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce4396-42f7-4a3a-b890-07356d9e86fd admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:41.380394 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef8d95c84244f19abac41398ffc88e9', 'request_id': 'req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:41.393302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8 admin admin] The token cache is being invalidate because project bef8d95c84244f19abac41398ffc88e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:41.394239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:41.395088 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4fcd1c1c-318e-44c3-810a-ccf5bff40cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:41.397578 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 855/1741] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:40 2022] DELETE /identity/v3/projects/bef8d95c84244f19abac41398ffc88e9 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:41.404196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330ba9e-33c9-4b3a-8a68-636accbf51f5 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:41.404600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330ba9e-33c9-4b3a-8a68-636accbf51f5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:41.404995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330ba9e-33c9-4b3a-8a68-636accbf51f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:41.430439 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 887/1742] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:41 2022] PUT /identity/v3/projects/c55500b03ab041979cd6d7e3a8bde147/users/b18cff690f7e408da812ce7f02f4b2d5/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:56:41.432166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fab24e-ee15-4c45-a90e-8269abce983e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:41.433118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fab24e-ee15-4c45-a90e-8269abce983e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:41.433503 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fab24e-ee15-4c45-a90e-8269abce983e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:41.470320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-3330ba9e-33c9-4b3a-8a68-636accbf51f5 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:41.472262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3330ba9e-33c9-4b3a-8a68-636accbf51f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:41.492369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-78fab24e-ee15-4c45-a90e-8269abce983e None None] MFA Rules not processed for user `b18cff690f7e408da812ce7f02f4b2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:41.493862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-78fab24e-ee15-4c45-a90e-8269abce983e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:41.571782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 856/1743] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:41.573268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9089a9-be91-4e0b-be3f-7fde22d3b10b None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:41.575937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9089a9-be91-4e0b-be3f-7fde22d3b10b None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:41.576180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9089a9-be91-4e0b-be3f-7fde22d3b10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:41.600365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ef9089a9-be91-4e0b-be3f-7fde22d3b10b None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:41.601821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ef9089a9-be91-4e0b-be3f-7fde22d3b10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:41.616783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 857/1744] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:41.618275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9089a9-be91-4e0b-be3f-7fde22d3b10b None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:41.632606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9089a9-be91-4e0b-be3f-7fde22d3b10b None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:41.633151 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 888/1745] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:41.637929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fab24e-ee15-4c45-a90e-8269abce983e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:41.639911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-78fab24e-ee15-4c45-a90e-8269abce983e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:41.651028 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f514ef0-cf93-4e50-b7f2-31204e59f802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:41.651799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f514ef0-cf93-4e50-b7f2-31204e59f802 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:41.652084 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f514ef0-cf93-4e50-b7f2-31204e59f802 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:41.652363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f514ef0-cf93-4e50-b7f2-31204e59f802 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:41.655127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f514ef0-cf93-4e50-b7f2-31204e59f802 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:41.655439 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f514ef0-cf93-4e50-b7f2-31204e59f802 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:41.658307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f514ef0-cf93-4e50-b7f2-31204e59f802 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:41.692336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fab24e-ee15-4c45-a90e-8269abce983e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:41.725761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177075f4-bda6-435a-abae-c73d2c971a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:41.726329 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177075f4-bda6-435a-abae-c73d2c971a0d admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:41.726641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177075f4-bda6-435a-abae-c73d2c971a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:41.726949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177075f4-bda6-435a-abae-c73d2c971a0d admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:41.731028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177075f4-bda6-435a-abae-c73d2c971a0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:41.736721 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177075f4-bda6-435a-abae-c73d2c971a0d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:41.741889 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 858/1746] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:41.779063 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f514ef0-cf93-4e50-b7f2-31204e59f802 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:41.811009 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f514ef0-cf93-4e50-b7f2-31204e59f802 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:41.828378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea4ed5-198f-4166-9975-f4f263d7d9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:41.829078 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea4ed5-198f-4166-9975-f4f263d7d9db admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:41.829367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea4ed5-198f-4166-9975-f4f263d7d9db admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:41.829638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea4ed5-198f-4166-9975-f4f263d7d9db admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:41.830280 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea4ed5-198f-4166-9975-f4f263d7d9db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:41.833289 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea4ed5-198f-4166-9975-f4f263d7d9db admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:41.912052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 889/1747] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:41 2022] POST /identity/v3/projects => generated 365 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:56:41.913209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177075f4-bda6-435a-abae-c73d2c971a0d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:41.935256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 859/1748] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:41 2022] POST /identity/v3/projects => generated 401 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:41.936833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea4ed5-198f-4166-9975-f4f263d7d9db admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:42.122544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177075f4-bda6-435a-abae-c73d2c971a0d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:42.147435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3ad14-a8d0-491a-8648-cfbae3901ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:42.151515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3ad14-a8d0-491a-8648-cfbae3901ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:42.151801 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3ad14-a8d0-491a-8648-cfbae3901ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:42.152137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3ad14-a8d0-491a-8648-cfbae3901ce9 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:42.154477 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea4ed5-198f-4166-9975-f4f263d7d9db admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:42.154859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3ad14-a8d0-491a-8648-cfbae3901ce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:42.160964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3ad14-a8d0-491a-8648-cfbae3901ce9 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:42.186708 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc7dad-a0a6-4aca-902f-6006ff346c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:42.187357 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc7dad-a0a6-4aca-902f-6006ff346c3c admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:42.192423 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc7dad-a0a6-4aca-902f-6006ff346c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:42.193074 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc7dad-a0a6-4aca-902f-6006ff346c3c admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:42.197300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc7dad-a0a6-4aca-902f-6006ff346c3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:42.201248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc7dad-a0a6-4aca-902f-6006ff346c3c admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:42.335975 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 890/1750] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:41 2022] POST /identity/v3/users => generated 289 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:56:42.336369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 860/1750] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:41 2022] POST /identity/v3/users => generated 301 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:56:42.338899 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc7dad-a0a6-4aca-902f-6006ff346c3c admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:42.343458 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3ad14-a8d0-491a-8648-cfbae3901ce9 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:42.401807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc7dad-a0a6-4aca-902f-6006ff346c3c admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:42.403867 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3ad14-a8d0-491a-8648-cfbae3901ce9 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:42.427486 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aa9af-794c-4503-9cee-8efbb7c806a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:42.428253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aa9af-794c-4503-9cee-8efbb7c806a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:42.430291 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fdf39-e74b-4995-b08d-ecf276a9e772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:42.431409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fdf39-e74b-4995-b08d-ecf276a9e772 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:42.431409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fdf39-e74b-4995-b08d-ecf276a9e772 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:42.431579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fdf39-e74b-4995-b08d-ecf276a9e772 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:42.432175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fdf39-e74b-4995-b08d-ecf276a9e772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:42.441251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aa9af-794c-4503-9cee-8efbb7c806a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:42.441870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aa9af-794c-4503-9cee-8efbb7c806a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:42.477332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180aa9af-794c-4503-9cee-8efbb7c806a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:42.477332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180aa9af-794c-4503-9cee-8efbb7c806a6 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:42.488442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fdf39-e74b-4995-b08d-ecf276a9e772 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:42.497513 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 861/1751] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:42.506185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aa9af-794c-4503-9cee-8efbb7c806a6 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:42.509348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-180aa9af-794c-4503-9cee-8efbb7c806a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:42.530568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 891/1752] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:42.533155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fdf39-e74b-4995-b08d-ecf276a9e772 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:42.560265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fdf39-e74b-4995-b08d-ecf276a9e772 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:42.575749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac765ae1-1fac-474f-9bd1-fb0b343a0092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:42.577062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac765ae1-1fac-474f-9bd1-fb0b343a0092 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:42.577791 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac765ae1-1fac-474f-9bd1-fb0b343a0092 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:42.578230 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac765ae1-1fac-474f-9bd1-fb0b343a0092 admin admin] PATH_INFO: `/v3/projects/b442dc12ef84446e9c181c935fedb397/users/75acc83621fb439f9c2eaf38e28ee8b6/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:42.584504 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:42.633536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac765ae1-1fac-474f-9bd1-fb0b343a0092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b442dc12ef84446e9c181c935fedb397, user_id=75acc83621fb439f9c2eaf38e28ee8b6, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:42.636269 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac765ae1-1fac-474f-9bd1-fb0b343a0092 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:42.700734 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aa9af-794c-4503-9cee-8efbb7c806a6 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:42.707780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2896884-6f50-4c90-9c20-99aca3ca505c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:42.708550 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2896884-6f50-4c90-9c20-99aca3ca505c service placement] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:42.708919 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2896884-6f50-4c90-9c20-99aca3ca505c service placement] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:42.709126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2896884-6f50-4c90-9c20-99aca3ca505c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:42.712260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f2896884-6f50-4c90-9c20-99aca3ca505c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:42.730066 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2896884-6f50-4c90-9c20-99aca3ca505c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:42.731100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2896884-6f50-4c90-9c20-99aca3ca505c service placement] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:42.747575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 862/1753] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:42.749123 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2896884-6f50-4c90-9c20-99aca3ca505c service placement] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:42.758097 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 892/1754] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:42 2022] PUT /identity/v3/projects/b442dc12ef84446e9c181c935fedb397/users/75acc83621fb439f9c2eaf38e28ee8b6/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:42.764699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac765ae1-1fac-474f-9bd1-fb0b343a0092 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:42.764938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2896884-6f50-4c90-9c20-99aca3ca505c service placement] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:42.786475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:42.868065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277977d-9e9d-49e6-8d1b-d625dec07088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:42.868065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277977d-9e9d-49e6-8d1b-d625dec07088 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:42.868065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277977d-9e9d-49e6-8d1b-d625dec07088 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:42.868065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277977d-9e9d-49e6-8d1b-d625dec07088 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:42.868407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277977d-9e9d-49e6-8d1b-d625dec07088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:42.868407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277977d-9e9d-49e6-8d1b-d625dec07088 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:42.902242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 863/1755] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:42.902242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277977d-9e9d-49e6-8d1b-d625dec07088 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:42.904127 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac765ae1-1fac-474f-9bd1-fb0b343a0092 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:42.912126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61448ce0-81b8-4844-a7b8-e1eb2c7cb6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:42.913471 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61448ce0-81b8-4844-a7b8-e1eb2c7cb6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:42.913471 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61448ce0-81b8-4844-a7b8-e1eb2c7cb6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:42.913471 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61448ce0-81b8-4844-a7b8-e1eb2c7cb6a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:42.913969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61448ce0-81b8-4844-a7b8-e1eb2c7cb6a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:42.922827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61448ce0-81b8-4844-a7b8-e1eb2c7cb6a7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:42.929139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277977d-9e9d-49e6-8d1b-d625dec07088 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:42.935987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3126c23-fe52-4773-a3dc-e76ee79904f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:42.942517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3126c23-fe52-4773-a3dc-e76ee79904f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:42.942517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3126c23-fe52-4773-a3dc-e76ee79904f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:42.942517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3126c23-fe52-4773-a3dc-e76ee79904f0 admin admin] PATH_INFO: `/v3/projects/2ff17079c32646609485abc17be4b127/users/76c65055367341ae91b53151f9c9102b/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:42.942767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 893/1756] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:42.945485 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61448ce0-81b8-4844-a7b8-e1eb2c7cb6a7 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:42.968389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3126c23-fe52-4773-a3dc-e76ee79904f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ff17079c32646609485abc17be4b127, user_id=76c65055367341ae91b53151f9c9102b, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:42.971321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3126c23-fe52-4773-a3dc-e76ee79904f0 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:42.989352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61448ce0-81b8-4844-a7b8-e1eb2c7cb6a7 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:43.007304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077d7cc-3d58-44bd-9e17-93660d1c3c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:43.008625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077d7cc-3d58-44bd-9e17-93660d1c3c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:43.008982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077d7cc-3d58-44bd-9e17-93660d1c3c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:43.009486 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077d7cc-3d58-44bd-9e17-93660d1c3c17 admin admin] PATH_INFO: `/v3/projects/b442dc12ef84446e9c181c935fedb397/users/75acc83621fb439f9c2eaf38e28ee8b6/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:43.013303 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077d7cc-3d58-44bd-9e17-93660d1c3c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b442dc12ef84446e9c181c935fedb397, user_id=75acc83621fb439f9c2eaf38e28ee8b6, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:43.022721 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077d7cc-3d58-44bd-9e17-93660d1c3c17 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:43.043363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 894/1757] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:42 2022] PUT /identity/v3/projects/b442dc12ef84446e9c181c935fedb397/users/75acc83621fb439f9c2eaf38e28ee8b6/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:56:43.057596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7cff2-9b5a-4fa1-993f-000c80a0604c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:43.057596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7cff2-9b5a-4fa1-993f-000c80a0604c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:43.057596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7cff2-9b5a-4fa1-993f-000c80a0604c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:43.089636 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 864/1758] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:42 2022] PUT /identity/v3/projects/2ff17079c32646609485abc17be4b127/users/76c65055367341ae91b53151f9c9102b/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:43.094733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d68ec-12f7-48c5-814c-c62e5abfb3c5 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:43.095219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d68ec-12f7-48c5-814c-c62e5abfb3c5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:43.095905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d68ec-12f7-48c5-814c-c62e5abfb3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:43.180205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-6c5d68ec-12f7-48c5-814c-c62e5abfb3c5 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:43.186861 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6c5d68ec-12f7-48c5-814c-c62e5abfb3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:43.191919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-41f7cff2-9b5a-4fa1-993f-000c80a0604c None None] MFA Rules not processed for user `75acc83621fb439f9c2eaf38e28ee8b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:43.197533 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-41f7cff2-9b5a-4fa1-993f-000c80a0604c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:43.340118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 865/1759] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:56:43.350058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d68ec-12f7-48c5-814c-c62e5abfb3c5 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:43.351917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6c5d68ec-12f7-48c5-814c-c62e5abfb3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:43.439408 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 895/1760] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:43.490443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d68ec-12f7-48c5-814c-c62e5abfb3c5 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:43.496198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00432f4-7811-472b-b974-6c344ec3e1c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:43.496875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00432f4-7811-472b-b974-6c344ec3e1c8 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:43.497156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00432f4-7811-472b-b974-6c344ec3e1c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:43.497429 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00432f4-7811-472b-b974-6c344ec3e1c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:43.517517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00432f4-7811-472b-b974-6c344ec3e1c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:43.520344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00432f4-7811-472b-b974-6c344ec3e1c8 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:43.549456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 866/1761] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:56:43.782523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7cff2-9b5a-4fa1-993f-000c80a0604c None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:43.786258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-41f7cff2-9b5a-4fa1-993f-000c80a0604c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:43.811995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8602ee-7619-40d1-93a4-abb9dbcad710 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:43.812412 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8602ee-7619-40d1-93a4-abb9dbcad710 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:43.813510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8602ee-7619-40d1-93a4-abb9dbcad710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:43.861771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ea8602ee-7619-40d1-93a4-abb9dbcad710 None None] MFA Rules not processed for user `63a2902f465f4fd1aed80060c2669259`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:43.866609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ea8602ee-7619-40d1-93a4-abb9dbcad710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:43.947125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7cff2-9b5a-4fa1-993f-000c80a0604c None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:43.955354 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be233d1-7eb1-4042-b000-70eb454eb75c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:43.956761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be233d1-7eb1-4042-b000-70eb454eb75c service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:43.957504 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be233d1-7eb1-4042-b000-70eb454eb75c service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:43.957948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be233d1-7eb1-4042-b000-70eb454eb75c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:43.984441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be233d1-7eb1-4042-b000-70eb454eb75c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:43.988004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be233d1-7eb1-4042-b000-70eb454eb75c service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:44.112853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 867/1762] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:56:44.145651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 896/1763] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:44.483095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:44.508531 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8602ee-7619-40d1-93a4-abb9dbcad710 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:44.511345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ea8602ee-7619-40d1-93a4-abb9dbcad710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:44.531945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8602ee-7619-40d1-93a4-abb9dbcad710 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:44.540384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738d80b-2877-4e68-b5af-be84ab644beb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:44.540384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738d80b-2877-4e68-b5af-be84ab644beb service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:44.541113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738d80b-2877-4e68-b5af-be84ab644beb service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:44.541682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738d80b-2877-4e68-b5af-be84ab644beb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:44.648854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:44.710135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738d80b-2877-4e68-b5af-be84ab644beb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:44.712958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738d80b-2877-4e68-b5af-be84ab644beb service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:44.772926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 868/1764] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:45.036856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd4156-82c7-416b-a2f3-c58e45c9b232 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:45.037395 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd4156-82c7-416b-a2f3-c58e45c9b232 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:45.037395 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd4156-82c7-416b-a2f3-c58e45c9b232 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:45.039441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 897/1765] 10.222.0.46 () {54 vars in 912 bytes} [Fri Sep 9 12:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:56:45.045519 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70adb8-b0b2-4142-934c-406fe882218f None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:45.045914 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70adb8-b0b2-4142-934c-406fe882218f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:45.046304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70adb8-b0b2-4142-934c-406fe882218f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:45.084391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-5a70adb8-b0b2-4142-934c-406fe882218f None None] MFA Rules not processed for user `e46bd9d49bf64013ae58770b0284d81e`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:45.086169 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5a70adb8-b0b2-4142-934c-406fe882218f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:45.102599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 869/1766] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 12:56:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:45.117649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd4156-82c7-416b-a2f3-c58e45c9b232 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:45.140162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd4156-82c7-416b-a2f3-c58e45c9b232 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:45.145737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76b4c2-bae4-46d9-a2d8-205eba43db5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:45.147551 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76b4c2-bae4-46d9-a2d8-205eba43db5a service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:45.148012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76b4c2-bae4-46d9-a2d8-205eba43db5a service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:45.148448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76b4c2-bae4-46d9-a2d8-205eba43db5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:45.158093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76b4c2-bae4-46d9-a2d8-205eba43db5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:45.160499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76b4c2-bae4-46d9-a2d8-205eba43db5a service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:45.183864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 898/1767] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:46.233494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20db76-a714-49f5-97f5-85727fb23ee6 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:46.234675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20db76-a714-49f5-97f5-85727fb23ee6 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:46.235152 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20db76-a714-49f5-97f5-85727fb23ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:46.301316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0414e2-399a-4053-9ae9-0571831b3c56 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:46.303489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-bd20db76-a714-49f5-97f5-85727fb23ee6 None None] MFA Rules not processed for user `6ec951604cde4768beb02906caf6d7c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:46.304753 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0414e2-399a-4053-9ae9-0571831b3c56 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:46.305210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0414e2-399a-4053-9ae9-0571831b3c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:46.306536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-bd20db76-a714-49f5-97f5-85727fb23ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:46.403634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-5c0414e2-399a-4053-9ae9-0571831b3c56 None None] MFA Rules not processed for user `76c65055367341ae91b53151f9c9102b`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:46.407264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5c0414e2-399a-4053-9ae9-0571831b3c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:46.500713 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 870/1768] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:46.599158 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 899/1769] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:46 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:46.604821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20db76-a714-49f5-97f5-85727fb23ee6 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:46.620398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20db76-a714-49f5-97f5-85727fb23ee6 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:46.639928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fec700-4a20-4fb7-90dc-3eb85a950394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:46.641836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fec700-4a20-4fb7-90dc-3eb85a950394 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:46.642395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fec700-4a20-4fb7-90dc-3eb85a950394 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:46.642987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fec700-4a20-4fb7-90dc-3eb85a950394 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:46.644154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fec700-4a20-4fb7-90dc-3eb85a950394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:46.649915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fec700-4a20-4fb7-90dc-3eb85a950394 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:46.925627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 871/1770] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:46 2022] POST /identity/v3/projects => generated 405 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:46.928243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0414e2-399a-4053-9ae9-0571831b3c56 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:46.980774 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0414e2-399a-4053-9ae9-0571831b3c56 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:47.044819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:47.179859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10a778-32b5-4e15-ad80-7607c7119af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:47.179859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10a778-32b5-4e15-ad80-7607c7119af7 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:47.179859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10a778-32b5-4e15-ad80-7607c7119af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:47.179859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10a778-32b5-4e15-ad80-7607c7119af7 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:47.180587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10a778-32b5-4e15-ad80-7607c7119af7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:47.180587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10a778-32b5-4e15-ad80-7607c7119af7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:47.290772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 900/1771] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:46 2022] POST /identity/v3/users => generated 309 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:47.296753 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fec700-4a20-4fb7-90dc-3eb85a950394 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:47.310976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fec700-4a20-4fb7-90dc-3eb85a950394 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:47.320339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252bda2-c2b6-4fa6-9c7c-492df1462fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:47.320339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252bda2-c2b6-4fa6-9c7c-492df1462fdc admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:47.320339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252bda2-c2b6-4fa6-9c7c-492df1462fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:47.320638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252bda2-c2b6-4fa6-9c7c-492df1462fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:47.321534 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252bda2-c2b6-4fa6-9c7c-492df1462fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:47.323465 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252bda2-c2b6-4fa6-9c7c-492df1462fdc admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:47.337482 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 872/1772] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:47.341652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10a778-32b5-4e15-ad80-7607c7119af7 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:47.353993 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10a778-32b5-4e15-ad80-7607c7119af7 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:47.404593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf8815-7df8-4538-aa7f-afcd126cd0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:47.404593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf8815-7df8-4538-aa7f-afcd126cd0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:47.404593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf8815-7df8-4538-aa7f-afcd126cd0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:47.404593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf8815-7df8-4538-aa7f-afcd126cd0c4 admin admin] PATH_INFO: `/v3/projects/0b87d4c556074bfda005a43677fa8956/users/125340d12b4142a5b96bf9e8a102d85d/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:47.440750 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf8815-7df8-4538-aa7f-afcd126cd0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b87d4c556074bfda005a43677fa8956, user_id=125340d12b4142a5b96bf9e8a102d85d, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:47.444594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf8815-7df8-4538-aa7f-afcd126cd0c4 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:47.699197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 901/1773] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:47 2022] PUT /identity/v3/projects/0b87d4c556074bfda005a43677fa8956/users/125340d12b4142a5b96bf9e8a102d85d/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:47.703763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252bda2-c2b6-4fa6-9c7c-492df1462fdc admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:47.724741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252bda2-c2b6-4fa6-9c7c-492df1462fdc admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:47.747646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:47.863461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aba6d2-03f2-428c-9a97-1f35bd626284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:47.864276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aba6d2-03f2-428c-9a97-1f35bd626284 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:47.864570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aba6d2-03f2-428c-9a97-1f35bd626284 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:47.864838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aba6d2-03f2-428c-9a97-1f35bd626284 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:47.865720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aba6d2-03f2-428c-9a97-1f35bd626284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:47.868929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aba6d2-03f2-428c-9a97-1f35bd626284 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:47.908011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 873/1774] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:56:47.908396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf8815-7df8-4538-aa7f-afcd126cd0c4 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:47.932843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf8815-7df8-4538-aa7f-afcd126cd0c4 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:47.975236 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb61074-ec7f-4f97-9f5c-b2d004e467d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:47.976017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb61074-ec7f-4f97-9f5c-b2d004e467d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:47.976347 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb61074-ec7f-4f97-9f5c-b2d004e467d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:47.977664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb61074-ec7f-4f97-9f5c-b2d004e467d0 admin admin] PATH_INFO: `/v3/projects/0b87d4c556074bfda005a43677fa8956/users/125340d12b4142a5b96bf9e8a102d85d/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:47.981092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb61074-ec7f-4f97-9f5c-b2d004e467d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b87d4c556074bfda005a43677fa8956, user_id=125340d12b4142a5b96bf9e8a102d85d, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:47.984321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb61074-ec7f-4f97-9f5c-b2d004e467d0 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:48.012064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 902/1775] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:47 2022] PUT /identity/v3/projects/0b87d4c556074bfda005a43677fa8956/users/125340d12b4142a5b96bf9e8a102d85d/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:49.037539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:49.850871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e055cf-d122-406a-bbe2-6f4f1401c066 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:49.851353 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e055cf-d122-406a-bbe2-6f4f1401c066 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:49.851353 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e055cf-d122-406a-bbe2-6f4f1401c066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:49.881719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-d7e055cf-d122-406a-bbe2-6f4f1401c066 None None] MFA Rules not processed for user `125340d12b4142a5b96bf9e8a102d85d`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:49.882984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d7e055cf-d122-406a-bbe2-6f4f1401c066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:49.940440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb61074-ec7f-4f97-9f5c-b2d004e467d0 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:49.961504 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 874/1776] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:49 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:56:50.007271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb61074-ec7f-4f97-9f5c-b2d004e467d0 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:50.012685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:50.016647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:50.016647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:50.016647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] PATH_INFO: `/v3/users/b18cff690f7e408da812ce7f02f4b2d5` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:50.016647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b18cff690f7e408da812ce7f02f4b2d5)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:50.017185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:50.285556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b18cff690f7e408da812ce7f02f4b2d5'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:50.409139 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b18cff690f7e408da812ce7f02f4b2d5'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:50.409784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b18cff690f7e408da812ce7f02f4b2d5'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:50.410406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b18cff690f7e408da812ce7f02f4b2d5'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:50.437114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b18cff690f7e408da812ce7f02f4b2d5'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:50.491704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 903/1777] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:49 2022] DELETE /identity/v3/users/b18cff690f7e408da812ce7f02f4b2d5 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:56:50.493411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d8ef1-a2ef-4ea4-8e22-88432e0a9175 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:50.493874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d8ef1-a2ef-4ea4-8e22-88432e0a9175 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:50.494298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d8ef1-a2ef-4ea4-8e22-88432e0a9175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:50.521825 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-767d8ef1-a2ef-4ea4-8e22-88432e0a9175 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:50.529128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-767d8ef1-a2ef-4ea4-8e22-88432e0a9175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:50.584955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:50.627991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 875/1778] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:50.637553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:50.639376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:50.719731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2729c-562d-4f22-a178-5b8c8a167930 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:50.720277 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2729c-562d-4f22-a178-5b8c8a167930 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:50.720783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2729c-562d-4f22-a178-5b8c8a167930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:50.744011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7fe3a-0eaa-4676-a08c-5a20b5a29b1a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:50.752493 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34855297-aa66-4e36-9d58-b3a609fd934c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:50.753660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-a0c2729c-562d-4f22-a178-5b8c8a167930 None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:50.754984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a0c2729c-562d-4f22-a178-5b8c8a167930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:50.756103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34855297-aa66-4e36-9d58-b3a609fd934c service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:50.756423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34855297-aa66-4e36-9d58-b3a609fd934c service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:50.757055 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34855297-aa66-4e36-9d58-b3a609fd934c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:50.783273 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34855297-aa66-4e36-9d58-b3a609fd934c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:50.785372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34855297-aa66-4e36-9d58-b3a609fd934c service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:50.798542 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 904/1779] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:50.851947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 876/1780] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 12:56:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:50.867751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34855297-aa66-4e36-9d58-b3a609fd934c service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:50.870415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-34855297-aa66-4e36-9d58-b3a609fd934c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:50.974682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34855297-aa66-4e36-9d58-b3a609fd934c service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:50.981170 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39edfe6-965c-4a2c-a0a4-e063cecbdb22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:50.981796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39edfe6-965c-4a2c-a0a4-e063cecbdb22 service swift] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:50.982078 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39edfe6-965c-4a2c-a0a4-e063cecbdb22 service swift] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:50.982905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39edfe6-965c-4a2c-a0a4-e063cecbdb22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:50.992275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39edfe6-965c-4a2c-a0a4-e063cecbdb22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:50.993949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39edfe6-965c-4a2c-a0a4-e063cecbdb22 service swift] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:51.006253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 905/1781] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 12:56:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:56:51.084093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f1f36-930e-4358-bf54-302073f5e2e1 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:51.084381 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f1f36-930e-4358-bf54-302073f5e2e1 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:51.084659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f1f36-930e-4358-bf54-302073f5e2e1 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:51.086022 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 877/1782] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 12:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:56:51.118585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2484c-fd16-487e-ad90-0d9c25760231 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:51.127208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2484c-fd16-487e-ad90-0d9c25760231 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:51.127208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2484c-fd16-487e-ad90-0d9c25760231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:51.164282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d6f2484c-fd16-487e-ad90-0d9c25760231 None None] MFA Rules not processed for user `33e177d86450489cacc86578ba26509d`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:51.169311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d6f2484c-fd16-487e-ad90-0d9c25760231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:51.350515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 906/1783] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 12:56:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:51.413599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f1f36-930e-4358-bf54-302073f5e2e1 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:51.416409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f34f1f36-930e-4358-bf54-302073f5e2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:51.450671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f1f36-930e-4358-bf54-302073f5e2e1 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:51.467564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1bfb5-3e0e-4e01-a641-aec9c855add6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:51.468544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1bfb5-3e0e-4e01-a641-aec9c855add6 service placement] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:51.469124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1bfb5-3e0e-4e01-a641-aec9c855add6 service placement] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:51.469555 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1bfb5-3e0e-4e01-a641-aec9c855add6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:51.483144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1bfb5-3e0e-4e01-a641-aec9c855add6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:51.488540 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1bfb5-3e0e-4e01-a641-aec9c855add6 service placement] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:51.506891 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 878/1784] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:51.842917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:52.033282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2484c-fd16-487e-ad90-0d9c25760231 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:52.035295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d6f2484c-fd16-487e-ad90-0d9c25760231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:52.054571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2484c-fd16-487e-ad90-0d9c25760231 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:52.065912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed591a30-1516-4dd6-ac17-2792a9cf6153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:52.067088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed591a30-1516-4dd6-ac17-2792a9cf6153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:52.067566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed591a30-1516-4dd6-ac17-2792a9cf6153 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:52.068099 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed591a30-1516-4dd6-ac17-2792a9cf6153 admin admin] PATH_INFO: `/v3/projects/c55500b03ab041979cd6d7e3a8bde147` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:52.069962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed591a30-1516-4dd6-ac17-2792a9cf6153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c55500b03ab041979cd6d7e3a8bde147)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:52.072162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed591a30-1516-4dd6-ac17-2792a9cf6153 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:52.116997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:52.320595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ed591a30-1516-4dd6-ac17-2792a9cf6153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c55500b03ab041979cd6d7e3a8bde147', 'request_id': 'req-ed591a30-1516-4dd6-ac17-2792a9cf6153'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:52.333167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ed591a30-1516-4dd6-ac17-2792a9cf6153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c55500b03ab041979cd6d7e3a8bde147', 'request_id': 'req-ed591a30-1516-4dd6-ac17-2792a9cf6153'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:52.429786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ed591a30-1516-4dd6-ac17-2792a9cf6153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c55500b03ab041979cd6d7e3a8bde147', 'request_id': 'req-ed591a30-1516-4dd6-ac17-2792a9cf6153'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:52.472412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ed591a30-1516-4dd6-ac17-2792a9cf6153 admin admin] The token cache is being invalidate because project c55500b03ab041979cd6d7e3a8bde147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:52.472980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ed591a30-1516-4dd6-ac17-2792a9cf6153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:52.473994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ed591a30-1516-4dd6-ac17-2792a9cf6153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:52.488902 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 907/1785] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:52 2022] DELETE /identity/v3/projects/c55500b03ab041979cd6d7e3a8bde147 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:52.491027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1bfb5-3e0e-4e01-a641-aec9c855add6 service placement] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:52.527770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1bfb5-3e0e-4e01-a641-aec9c855add6 service placement] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:52.688129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:52.689394 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:52.690707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:52.691447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] PATH_INFO: `/v3/users/75acc83621fb439f9c2eaf38e28ee8b6` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:52.695871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75acc83621fb439f9c2eaf38e28ee8b6)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:52.702700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:52.735482 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855b6f0-6b98-40a6-8cca-c56886f361cb None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:52.736217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855b6f0-6b98-40a6-8cca-c56886f361cb None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:52.736715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855b6f0-6b98-40a6-8cca-c56886f361cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:52.786441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-c855b6f0-6b98-40a6-8cca-c56886f361cb None None] MFA Rules not processed for user `e46bd9d49bf64013ae58770b0284d81e`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:52.788524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c855b6f0-6b98-40a6-8cca-c56886f361cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:52.884469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 908/1786] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Sep 9 12:56:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:56:52.973028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855b6f0-6b98-40a6-8cca-c56886f361cb None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:52.974997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c855b6f0-6b98-40a6-8cca-c56886f361cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:53.023872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855b6f0-6b98-40a6-8cca-c56886f361cb None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:53.039780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7dac5-33c5-4890-ad02-97df78e580f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:53.043863 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7dac5-33c5-4890-ad02-97df78e580f5 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:53.044549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75acc83621fb439f9c2eaf38e28ee8b6'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:53.047251 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7dac5-33c5-4890-ad02-97df78e580f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:53.048125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7dac5-33c5-4890-ad02-97df78e580f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:53.107886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7dac5-33c5-4890-ad02-97df78e580f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:53.110384 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7dac5-33c5-4890-ad02-97df78e580f5 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:53.116976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75acc83621fb439f9c2eaf38e28ee8b6'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:53.129209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75acc83621fb439f9c2eaf38e28ee8b6'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:53.130842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75acc83621fb439f9c2eaf38e28ee8b6'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:53.144910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 909/1787] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:56:53.146029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7dac5-33c5-4890-ad02-97df78e580f5 service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:53.152721 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d9d7dac5-33c5-4890-ad02-97df78e580f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:53.165801 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75acc83621fb439f9c2eaf38e28ee8b6'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:53.230746 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7dac5-33c5-4890-ad02-97df78e580f5 service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:53.233150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 879/1788] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:52 2022] DELETE /identity/v3/users/75acc83621fb439f9c2eaf38e28ee8b6 => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:53.244545 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:53.254143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:53.254492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:53.254784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] PATH_INFO: `/v3/users/76c65055367341ae91b53151f9c9102b` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:53.259448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] RBAC: Authorizing `identity:delete_user(user_id=76c65055367341ae91b53151f9c9102b)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:53.263873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:53.529821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:53.588811 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76c65055367341ae91b53151f9c9102b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:53.859221 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76c65055367341ae91b53151f9c9102b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:53.863219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76c65055367341ae91b53151f9c9102b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:53.863219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76c65055367341ae91b53151f9c9102b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:53.883625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76c65055367341ae91b53151f9c9102b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:53.935640 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 910/1789] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:53 2022] DELETE /identity/v3/users/76c65055367341ae91b53151f9c9102b => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:54.036195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:54.038999 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:54.113049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:54.166077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:54.168306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:54.251026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569b65b-11a7-4e8f-9e4a-b0372bf88054 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:54.381326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3b083-8ed9-433f-9c2a-92af09b7383a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:54.408668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:54.410293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:54.410614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:54.410897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] PATH_INFO: `/v3/users/eee8eee033184e9d9fc3f6cb8a802a70` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:54.419492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee8eee033184e9d9fc3f6cb8a802a70)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:54.421510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:54.454674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:54.499749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee46599-520f-42cf-81a6-49ecedb80639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:54.500673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee46599-520f-42cf-81a6-49ecedb80639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:54.501118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee46599-520f-42cf-81a6-49ecedb80639 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:54.501453 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee46599-520f-42cf-81a6-49ecedb80639 admin admin] PATH_INFO: `/v3/projects/b442dc12ef84446e9c181c935fedb397` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:54.505093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee46599-520f-42cf-81a6-49ecedb80639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b442dc12ef84446e9c181c935fedb397)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:54.510310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee46599-520f-42cf-81a6-49ecedb80639 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:54.606539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee8eee033184e9d9fc3f6cb8a802a70'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:54.625628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:54.700688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee8eee033184e9d9fc3f6cb8a802a70'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:54.704115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee8eee033184e9d9fc3f6cb8a802a70'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:54.706765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee8eee033184e9d9fc3f6cb8a802a70'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:54.716468 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1ee46599-520f-42cf-81a6-49ecedb80639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b442dc12ef84446e9c181c935fedb397', 'request_id': 'req-1ee46599-520f-42cf-81a6-49ecedb80639'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:54.730420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1ee46599-520f-42cf-81a6-49ecedb80639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b442dc12ef84446e9c181c935fedb397', 'request_id': 'req-1ee46599-520f-42cf-81a6-49ecedb80639'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:54.756432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee8eee033184e9d9fc3f6cb8a802a70'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:54.793540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 911/1790] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:54 2022] DELETE /identity/v3/users/eee8eee033184e9d9fc3f6cb8a802a70 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:56:54.796935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:54.799145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:54.853005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1ee46599-520f-42cf-81a6-49ecedb80639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b442dc12ef84446e9c181c935fedb397', 'request_id': 'req-1ee46599-520f-42cf-81a6-49ecedb80639'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:54.875037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1ee46599-520f-42cf-81a6-49ecedb80639 admin admin] The token cache is being invalidate because project b442dc12ef84446e9c181c935fedb397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:54.875647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1ee46599-520f-42cf-81a6-49ecedb80639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:54.877288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1ee46599-520f-42cf-81a6-49ecedb80639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:54.879913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 880/1791] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:54 2022] DELETE /identity/v3/projects/b442dc12ef84446e9c181c935fedb397 => generated 0 bytes in 845 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:54.882179 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285cba8-9f05-4efa-9876-20fd85fd4d36 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:54.882799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285cba8-9f05-4efa-9876-20fd85fd4d36 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:54.883324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285cba8-9f05-4efa-9876-20fd85fd4d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:54.958458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-a285cba8-9f05-4efa-9876-20fd85fd4d36 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:54.958458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a285cba8-9f05-4efa-9876-20fd85fd4d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:54.996300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 881/1792] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:54.996300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecda9f-d62b-42aa-93ac-b86c65477ed0 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:54.996300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecda9f-d62b-42aa-93ac-b86c65477ed0 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:54.996300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecda9f-d62b-42aa-93ac-b86c65477ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:55.011805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1099235-0083-4303-a51b-25c8197e6bbb admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:55.024776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:55.025492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:55.025771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:55.026039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] PATH_INFO: `/v3/projects/2ff17079c32646609485abc17be4b127` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:55.028019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ff17079c32646609485abc17be4b127)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:55.030079 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:55.034355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-fcecda9f-d62b-42aa-93ac-b86c65477ed0 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:55.035906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-fcecda9f-d62b-42aa-93ac-b86c65477ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:55.046548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 882/1793] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:55.047603 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecda9f-d62b-42aa-93ac-b86c65477ed0 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:55.055605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-fcecda9f-d62b-42aa-93ac-b86c65477ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:55.095361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:55.195153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:55.228085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ff17079c32646609485abc17be4b127', 'request_id': 'req-4b1e1c8a-91ca-4f16-927c-b44b0becade2'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:55.251381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ff17079c32646609485abc17be4b127', 'request_id': 'req-4b1e1c8a-91ca-4f16-927c-b44b0becade2'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:55.286476 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecda9f-d62b-42aa-93ac-b86c65477ed0 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:55.385059 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac8685-60ce-4b7f-b8f6-4630f965e9db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:55.385715 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac8685-60ce-4b7f-b8f6-4630f965e9db service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:55.385998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac8685-60ce-4b7f-b8f6-4630f965e9db service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:55.386273 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac8685-60ce-4b7f-b8f6-4630f965e9db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:55.483631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac8685-60ce-4b7f-b8f6-4630f965e9db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:55.485094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac8685-60ce-4b7f-b8f6-4630f965e9db service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:55.498314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 883/1794] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:55.504881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e083fe5-f381-4f1c-a68d-0c1ca02fdf0d None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:55.505212 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e083fe5-f381-4f1c-a68d-0c1ca02fdf0d None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:55.505751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e083fe5-f381-4f1c-a68d-0c1ca02fdf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:55.509796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ff17079c32646609485abc17be4b127', 'request_id': 'req-4b1e1c8a-91ca-4f16-927c-b44b0becade2'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:55.524335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] The token cache is being invalidate because project 2ff17079c32646609485abc17be4b127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:55.524877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:55.525696 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:55.527558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 912/1795] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:54 2022] DELETE /identity/v3/projects/2ff17079c32646609485abc17be4b127 => generated 0 bytes in 733 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:55.530774 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-5e083fe5-f381-4f1c-a68d-0c1ca02fdf0d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:55.531564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5e083fe5-f381-4f1c-a68d-0c1ca02fdf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:55.532035 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:55.533619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:55.547493 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e1c8a-91ca-4f16-927c-b44b0becade2 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:55.554400 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc314f2-f9e8-43e6-984c-92e75de6935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:55.555414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc314f2-f9e8-43e6-984c-92e75de6935b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:55.555854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc314f2-f9e8-43e6-984c-92e75de6935b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:55.556299 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc314f2-f9e8-43e6-984c-92e75de6935b admin admin] PATH_INFO: `/v3/users/125340d12b4142a5b96bf9e8a102d85d` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:55.557833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc314f2-f9e8-43e6-984c-92e75de6935b admin admin] RBAC: Authorizing `identity:delete_user(user_id=125340d12b4142a5b96bf9e8a102d85d)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:55.560182 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc314f2-f9e8-43e6-984c-92e75de6935b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:55.564061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 884/1796] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:55.569587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e083fe5-f381-4f1c-a68d-0c1ca02fdf0d None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:55.579113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e083fe5-f381-4f1c-a68d-0c1ca02fdf0d None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:55.585335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e3e75-5db2-457d-99b7-fd822e5fd7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:55.586168 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e3e75-5db2-457d-99b7-fd822e5fd7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:55.586638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e3e75-5db2-457d-99b7-fd822e5fd7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:55.587118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e3e75-5db2-457d-99b7-fd822e5fd7f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:55.587869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e3e75-5db2-457d-99b7-fd822e5fd7f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:55.588359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e3e75-5db2-457d-99b7-fd822e5fd7f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:55.590903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e3e75-5db2-457d-99b7-fd822e5fd7f4 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:55.630522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 885/1797] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:56:55.634038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e3e75-5db2-457d-99b7-fd822e5fd7f4 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:55.653785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e3e75-5db2-457d-99b7-fd822e5fd7f4 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:55.656816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444621ca-1cf3-43ef-8ebc-5309d1eb378b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:55.657721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444621ca-1cf3-43ef-8ebc-5309d1eb378b admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:55.658203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444621ca-1cf3-43ef-8ebc-5309d1eb378b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:55.658776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444621ca-1cf3-43ef-8ebc-5309d1eb378b admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:55.659860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444621ca-1cf3-43ef-8ebc-5309d1eb378b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:55.662021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444621ca-1cf3-43ef-8ebc-5309d1eb378b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:55.880674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 886/1798] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:55 2022] POST /identity/v3/projects => generated 369 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:56:55.887432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1dbca-4b99-42a0-acc2-4b3ce20bb84c None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:55.887722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1dbca-4b99-42a0-acc2-4b3ce20bb84c None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:55.887992 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1dbca-4b99-42a0-acc2-4b3ce20bb84c None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:55.889494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 887/1799] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 12:56:55 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:56:55.890086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1dbca-4b99-42a0-acc2-4b3ce20bb84c None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:55.964618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7bc314f2-f9e8-43e6-984c-92e75de6935b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '125340d12b4142a5b96bf9e8a102d85d'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:56.023318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1dbca-4b99-42a0-acc2-4b3ce20bb84c None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:56.028208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f13125-9123-4ffb-ad97-725f4bdaab4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:56.028881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f13125-9123-4ffb-ad97-725f4bdaab4f admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:56.029200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f13125-9123-4ffb-ad97-725f4bdaab4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:56.029461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f13125-9123-4ffb-ad97-725f4bdaab4f admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:56.030102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f13125-9123-4ffb-ad97-725f4bdaab4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:56.032267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f13125-9123-4ffb-ad97-725f4bdaab4f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:56.063833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7bc314f2-f9e8-43e6-984c-92e75de6935b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '125340d12b4142a5b96bf9e8a102d85d'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:56.064554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7bc314f2-f9e8-43e6-984c-92e75de6935b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '125340d12b4142a5b96bf9e8a102d85d'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:56.066359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7bc314f2-f9e8-43e6-984c-92e75de6935b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '125340d12b4142a5b96bf9e8a102d85d'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:56.094270 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7bc314f2-f9e8-43e6-984c-92e75de6935b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '125340d12b4142a5b96bf9e8a102d85d'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:56.123923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 913/1800] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:55 2022] DELETE /identity/v3/users/125340d12b4142a5b96bf9e8a102d85d => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:56.126438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ca23a-7199-4ce1-8a31-b70412b37769 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:56.127217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ca23a-7199-4ce1-8a31-b70412b37769 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:56.128339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ca23a-7199-4ce1-8a31-b70412b37769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:56.161923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-7e4ca23a-7199-4ce1-8a31-b70412b37769 None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:56.163370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7e4ca23a-7199-4ce1-8a31-b70412b37769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:56.183961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 888/1801] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:55 2022] POST /identity/v3/users => generated 285 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:56.183961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f13125-9123-4ffb-ad97-725f4bdaab4f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:56.186043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-22f13125-9123-4ffb-ad97-725f4bdaab4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:56.200515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f13125-9123-4ffb-ad97-725f4bdaab4f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:56.316563 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 914/1802] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 12:56:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:56.317486 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ca23a-7199-4ce1-8a31-b70412b37769 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:56.319262 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7e4ca23a-7199-4ce1-8a31-b70412b37769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:56.425091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:56.425839 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:56.426130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:56.430921 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] PATH_INFO: `/v3/projects/9073b1c2abbb43999ae2d9fc7023d338` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:56.430921 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9073b1c2abbb43999ae2d9fc7023d338)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:56.431544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:56.523394 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ca23a-7199-4ce1-8a31-b70412b37769 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:56.529105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c39edc-c1fa-4e17-a5ef-6680b9cc8152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:56.529725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c39edc-c1fa-4e17-a5ef-6680b9cc8152 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:56.530137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c39edc-c1fa-4e17-a5ef-6680b9cc8152 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:56.530413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c39edc-c1fa-4e17-a5ef-6680b9cc8152 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:56.531086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c39edc-c1fa-4e17-a5ef-6680b9cc8152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:56.532714 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c39edc-c1fa-4e17-a5ef-6680b9cc8152 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:56.541377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:56.547004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 915/1803] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:56.551447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c39edc-c1fa-4e17-a5ef-6680b9cc8152 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:56.553267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a3c39edc-c1fa-4e17-a5ef-6680b9cc8152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:56.571027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c39edc-c1fa-4e17-a5ef-6680b9cc8152 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:56.581100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e299c-ee98-448a-9b0e-90ddc1006393 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:56.581671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e299c-ee98-448a-9b0e-90ddc1006393 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:56.582258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e299c-ee98-448a-9b0e-90ddc1006393 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:56.582863 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e299c-ee98-448a-9b0e-90ddc1006393 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:56.595994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e299c-ee98-448a-9b0e-90ddc1006393 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:56.599225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e299c-ee98-448a-9b0e-90ddc1006393 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:56.621852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 916/1804] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:56.622849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e299c-ee98-448a-9b0e-90ddc1006393 service nova] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:56.639347 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e299c-ee98-448a-9b0e-90ddc1006393 service nova] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:56.648474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9073b1c2abbb43999ae2d9fc7023d338', 'request_id': 'req-2e89ede0-5c32-4c88-add7-97984303cc99'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:56.656732 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9073b1c2abbb43999ae2d9fc7023d338', 'request_id': 'req-2e89ede0-5c32-4c88-add7-97984303cc99'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:56.763978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9073b1c2abbb43999ae2d9fc7023d338', 'request_id': 'req-2e89ede0-5c32-4c88-add7-97984303cc99'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:56.776553 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] The token cache is being invalidate because project 9073b1c2abbb43999ae2d9fc7023d338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:56.776857 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:56.777843 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:56.779801 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 889/1805] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:56 2022] DELETE /identity/v3/projects/9073b1c2abbb43999ae2d9fc7023d338 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:56.780312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:56.781645 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:56.792995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3e53-5371-4992-845d-75121660bbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:56.793940 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3e53-5371-4992-845d-75121660bbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:56.794219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3e53-5371-4992-845d-75121660bbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:56.794655 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3e53-5371-4992-845d-75121660bbe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:56.795321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c3e53-5371-4992-845d-75121660bbe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:56.796975 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c3e53-5371-4992-845d-75121660bbe7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:56.812331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 917/1806] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:56.812331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3e53-5371-4992-845d-75121660bbe7 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:56.812331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-af7c3e53-5371-4992-845d-75121660bbe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:56.832444 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3e53-5371-4992-845d-75121660bbe7 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:56.838210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:56.838913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:56.839282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:56.839452 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] PATH_INFO: `/v3/users/6c24badd5d4844ae96312710474a8443` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:56.840665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c24badd5d4844ae96312710474a8443)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:56.842665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:56.890996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89ede0-5c32-4c88-add7-97984303cc99 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:56.901071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:56.901826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:56.902219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:56.902508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d admin admin] PATH_INFO: `/v3/projects/0b87d4c556074bfda005a43677fa8956` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:56.903879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b87d4c556074bfda005a43677fa8956)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:56.905844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:57.000733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:57.138209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b87d4c556074bfda005a43677fa8956', 'request_id': 'req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:57.138910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c24badd5d4844ae96312710474a8443'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:57.145222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b87d4c556074bfda005a43677fa8956', 'request_id': 'req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:57.206959 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c24badd5d4844ae96312710474a8443'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:57.207968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c24badd5d4844ae96312710474a8443'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:57.208677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c24badd5d4844ae96312710474a8443'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:57.229974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c24badd5d4844ae96312710474a8443'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:57.253882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 918/1807] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:56 2022] DELETE /identity/v3/users/6c24badd5d4844ae96312710474a8443 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:57.253882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:57.253882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:57.259247 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b87d4c556074bfda005a43677fa8956', 'request_id': 'req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:57.296816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d admin admin] The token cache is being invalidate because project 0b87d4c556074bfda005a43677fa8956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:57.298111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:57.305804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-7fdfad0e-c8bc-4ac5-b54f-5e05720c1b4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:57.307728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 890/1808] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:56 2022] DELETE /identity/v3/projects/0b87d4c556074bfda005a43677fa8956 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:57.320083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48faa84-1601-405c-a8d7-93030cf25575 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:57.320372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48faa84-1601-405c-a8d7-93030cf25575 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:57.320710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48faa84-1601-405c-a8d7-93030cf25575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:57.367826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-e48faa84-1601-405c-a8d7-93030cf25575 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:57.369067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e48faa84-1601-405c-a8d7-93030cf25575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:57.406944 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022cc15-e4fd-4fbd-b8e3-75f4bbbacbca admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:57.427413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cdca8-ddb1-4edb-b918-b10688a49952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:57.428200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cdca8-ddb1-4edb-b918-b10688a49952 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:57.428550 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cdca8-ddb1-4edb-b918-b10688a49952 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:57.428929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cdca8-ddb1-4edb-b918-b10688a49952 admin admin] PATH_INFO: `/v3/projects/80ea9150fd3845a799a5f90bfc893351/users/cb75c3a815494c3ea3153d01866f044b/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:57.474572 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cdca8-ddb1-4edb-b918-b10688a49952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ea9150fd3845a799a5f90bfc893351, user_id=cb75c3a815494c3ea3153d01866f044b, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:57.477557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cdca8-ddb1-4edb-b918-b10688a49952 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:57.509495 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 891/1809] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:57.516836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05413e-ac73-4b1d-b125-c7fcfe41a292 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:57.517538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05413e-ac73-4b1d-b125-c7fcfe41a292 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:57.517538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05413e-ac73-4b1d-b125-c7fcfe41a292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:57.559926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ac05413e-ac73-4b1d-b125-c7fcfe41a292 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:57.561673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ac05413e-ac73-4b1d-b125-c7fcfe41a292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:57.575805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 919/1810] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:57 2022] PUT /identity/v3/projects/80ea9150fd3845a799a5f90bfc893351/users/cb75c3a815494c3ea3153d01866f044b/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:57.591574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee058ad-7742-4f3a-ad95-cda91121a4f5 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:57.591999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee058ad-7742-4f3a-ad95-cda91121a4f5 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:57.592385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee058ad-7742-4f3a-ad95-cda91121a4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:57.670571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-dee058ad-7742-4f3a-ad95-cda91121a4f5 None None] MFA Rules not processed for user `cb75c3a815494c3ea3153d01866f044b`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:57.671593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-dee058ad-7742-4f3a-ad95-cda91121a4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:57.722735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 892/1811] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:56:57.733807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05413e-ac73-4b1d-b125-c7fcfe41a292 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:57.771427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05413e-ac73-4b1d-b125-c7fcfe41a292 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:57.803300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:57.807805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f225b74-c024-472b-b7e3-82a873f8d183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:57.808468 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225b74-c024-472b-b7e3-82a873f8d183 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:57.808742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225b74-c024-472b-b7e3-82a873f8d183 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:57.809161 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225b74-c024-472b-b7e3-82a873f8d183 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:57.809949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f225b74-c024-472b-b7e3-82a873f8d183 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:57.810957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f225b74-c024-472b-b7e3-82a873f8d183 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:57.813296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f225b74-c024-472b-b7e3-82a873f8d183 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:57.823700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 920/1812] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:57.832515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee058ad-7742-4f3a-ad95-cda91121a4f5 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:57.834418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-dee058ad-7742-4f3a-ad95-cda91121a4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:57.861789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 893/1813] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:57.864248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee058ad-7742-4f3a-ad95-cda91121a4f5 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:57.867831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f225b74-c024-472b-b7e3-82a873f8d183 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:57.870698 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4663b7-dd4a-40db-95fa-2d2a6d10be50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:57.871433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4663b7-dd4a-40db-95fa-2d2a6d10be50 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:57.871986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4663b7-dd4a-40db-95fa-2d2a6d10be50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:57.872707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4663b7-dd4a-40db-95fa-2d2a6d10be50 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:57.873524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4663b7-dd4a-40db-95fa-2d2a6d10be50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:57.875948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4663b7-dd4a-40db-95fa-2d2a6d10be50 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:57.887525 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f225b74-c024-472b-b7e3-82a873f8d183 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:57.894329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb181ea8-b512-48f4-95c3-33d1199345ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:57.895043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb181ea8-b512-48f4-95c3-33d1199345ea admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:57.895641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb181ea8-b512-48f4-95c3-33d1199345ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:57.895641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb181ea8-b512-48f4-95c3-33d1199345ea admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:57.896602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb181ea8-b512-48f4-95c3-33d1199345ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:57.898370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb181ea8-b512-48f4-95c3-33d1199345ea admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:58.058136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 894/1814] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:57 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:58.062149 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb181ea8-b512-48f4-95c3-33d1199345ea admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:58.064061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-eb181ea8-b512-48f4-95c3-33d1199345ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:58.070793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 921/1815] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:57 2022] POST /identity/v3/projects => generated 367 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:58.082886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4663b7-dd4a-40db-95fa-2d2a6d10be50 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:58.137485 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:58.197234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb181ea8-b512-48f4-95c3-33d1199345ea admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:58.212889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4663b7-dd4a-40db-95fa-2d2a6d10be50 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:58.221913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b4c6b-3a58-4102-8693-760a89519160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:58.231366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b4c6b-3a58-4102-8693-760a89519160 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:58.232285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b4c6b-3a58-4102-8693-760a89519160 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:58.232285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b4c6b-3a58-4102-8693-760a89519160 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:58.232733 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b4c6b-3a58-4102-8693-760a89519160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:58.234711 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b4c6b-3a58-4102-8693-760a89519160 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:58.272288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:58.272905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:58.273185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:58.273459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] PATH_INFO: `/v3/projects/1bf77868b1b84c7bb97e5ecd9348b00b` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:58.274621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bf77868b1b84c7bb97e5ecd9348b00b)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:58.276459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:58.371202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 922/1816] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:58 2022] POST /identity/v3/users => generated 290 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:58.372158 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b4c6b-3a58-4102-8693-760a89519160 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:58.383494 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b4c6b-3a58-4102-8693-760a89519160 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:58.389556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed67b7-87ae-4dbb-8a70-b93cec5510e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:58.390215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed67b7-87ae-4dbb-8a70-b93cec5510e8 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:58.390505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed67b7-87ae-4dbb-8a70-b93cec5510e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:58.391768 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed67b7-87ae-4dbb-8a70-b93cec5510e8 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:58.392492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed67b7-87ae-4dbb-8a70-b93cec5510e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:58.394723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed67b7-87ae-4dbb-8a70-b93cec5510e8 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:58.441129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf77868b1b84c7bb97e5ecd9348b00b', 'request_id': 'req-bce6225a-35b3-4e21-9347-c1db66f27f5a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:58.447872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf77868b1b84c7bb97e5ecd9348b00b', 'request_id': 'req-bce6225a-35b3-4e21-9347-c1db66f27f5a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:58.534125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf77868b1b84c7bb97e5ecd9348b00b', 'request_id': 'req-bce6225a-35b3-4e21-9347-c1db66f27f5a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:58.546594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 923/1817] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:58 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:58.546594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed67b7-87ae-4dbb-8a70-b93cec5510e8 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:58.549177 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-81ed67b7-87ae-4dbb-8a70-b93cec5510e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:58.564232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] The token cache is being invalidate because project 1bf77868b1b84c7bb97e5ecd9348b00b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:56:58.564720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:58.567386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:58.571095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed67b7-87ae-4dbb-8a70-b93cec5510e8 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:58.575458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 895/1818] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:56:58 2022] DELETE /identity/v3/projects/1bf77868b1b84c7bb97e5ecd9348b00b => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:58.575458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:58.577868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:58.675675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6225a-35b3-4e21-9347-c1db66f27f5a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:58.681384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaac8cc-0af7-49ea-9c09-2c02491731ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:58.682135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaac8cc-0af7-49ea-9c09-2c02491731ae admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:58.682508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaac8cc-0af7-49ea-9c09-2c02491731ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:58.683001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaac8cc-0af7-49ea-9c09-2c02491731ae admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:58.683782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaac8cc-0af7-49ea-9c09-2c02491731ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:58.685497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaac8cc-0af7-49ea-9c09-2c02491731ae admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:58.698534 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 896/1819] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:58.699463 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaac8cc-0af7-49ea-9c09-2c02491731ae admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:58.701840 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7aaac8cc-0af7-49ea-9c09-2c02491731ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:58.716692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaac8cc-0af7-49ea-9c09-2c02491731ae admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:58.723483 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb638bf-0afe-4d01-aa5b-22956d7bd486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:58.729075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb638bf-0afe-4d01-aa5b-22956d7bd486 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:58.729075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb638bf-0afe-4d01-aa5b-22956d7bd486 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:58.729075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb638bf-0afe-4d01-aa5b-22956d7bd486 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:58.729075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb638bf-0afe-4d01-aa5b-22956d7bd486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:58.729075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb638bf-0afe-4d01-aa5b-22956d7bd486 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:58.751996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 897/1820] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:58.757545 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:58.757814 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c34dfa-9a05-4fbe-8aaa-5e2ca56401c7 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:58.757814 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c34dfa-9a05-4fbe-8aaa-5e2ca56401c7 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:58.757965 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:58.758040 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c34dfa-9a05-4fbe-8aaa-5e2ca56401c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:58.758113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:58.758113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] PATH_INFO: `/v3/users/274c5b2c58934a3eb3a9226d13bf01d4` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:58.763366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=274c5b2c58934a3eb3a9226d13bf01d4)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:58.768309 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:58.791558 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-b5c34dfa-9a05-4fbe-8aaa-5e2ca56401c7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:58.792980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b5c34dfa-9a05-4fbe-8aaa-5e2ca56401c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:58.800568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:58.805622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 898/1821] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:58.809255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c34dfa-9a05-4fbe-8aaa-5e2ca56401c7 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:58.820206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c34dfa-9a05-4fbe-8aaa-5e2ca56401c7 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:58.825430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d2de1-a70c-4ee6-b9cc-210f334e4f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:58.826108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d2de1-a70c-4ee6-b9cc-210f334e4f7d admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:58.826457 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d2de1-a70c-4ee6-b9cc-210f334e4f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:58.834665 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d2de1-a70c-4ee6-b9cc-210f334e4f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:58.834665 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d2de1-a70c-4ee6-b9cc-210f334e4f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:58.834665 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d2de1-a70c-4ee6-b9cc-210f334e4f7d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:58.842204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 899/1822] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:58.843154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d2de1-a70c-4ee6-b9cc-210f334e4f7d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:58.854059 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d2de1-a70c-4ee6-b9cc-210f334e4f7d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:58.858994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282acf4-3633-4af6-83a7-c6e38868b50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:58.859647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282acf4-3633-4af6-83a7-c6e38868b50b admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:58.859929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282acf4-3633-4af6-83a7-c6e38868b50b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:58.860209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282acf4-3633-4af6-83a7-c6e38868b50b admin admin] PATH_INFO: `/v3/projects/8556d6face314983b0a162018c116d63/users/c1c8801d2d9246c2840cea98975184c8/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:58.881877 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282acf4-3633-4af6-83a7-c6e38868b50b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8556d6face314983b0a162018c116d63, user_id=c1c8801d2d9246c2840cea98975184c8, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:58.884830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282acf4-3633-4af6-83a7-c6e38868b50b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:59.008373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '274c5b2c58934a3eb3a9226d13bf01d4'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:59.019667 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 900/1823] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:58 2022] PUT /identity/v3/projects/8556d6face314983b0a162018c116d63/users/c1c8801d2d9246c2840cea98975184c8/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:59.021047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717ad22-dfa8-4393-bd00-9db3e93bc427 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:59.021337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717ad22-dfa8-4393-bd00-9db3e93bc427 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:59.021611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717ad22-dfa8-4393-bd00-9db3e93bc427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:59.043445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-2717ad22-dfa8-4393-bd00-9db3e93bc427 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:59.046619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2717ad22-dfa8-4393-bd00-9db3e93bc427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:59.114286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 901/1824] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:59.115941 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717ad22-dfa8-4393-bd00-9db3e93bc427 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:59.128844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '274c5b2c58934a3eb3a9226d13bf01d4'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:59.131280 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717ad22-dfa8-4393-bd00-9db3e93bc427 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:59.131401 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '274c5b2c58934a3eb3a9226d13bf01d4'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:59.131505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '274c5b2c58934a3eb3a9226d13bf01d4'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:59.136017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fe53a-d7ee-4036-b76e-2893b3a40652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:59.136950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fe53a-d7ee-4036-b76e-2893b3a40652 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:59.137254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fe53a-d7ee-4036-b76e-2893b3a40652 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:59.137539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fe53a-d7ee-4036-b76e-2893b3a40652 admin admin] PATH_INFO: `/v3/projects/3a265ce8247f4c3aac708aff32f44df9/users/4cc9d272d6714ad7aac0e208a8ed1175/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:59.154376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fe53a-d7ee-4036-b76e-2893b3a40652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a265ce8247f4c3aac708aff32f44df9, user_id=4cc9d272d6714ad7aac0e208a8ed1175, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:59.157057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fe53a-d7ee-4036-b76e-2893b3a40652 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:59.158522 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '274c5b2c58934a3eb3a9226d13bf01d4'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:56:59.172694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 924/1825] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:56:58 2022] DELETE /identity/v3/users/274c5b2c58934a3eb3a9226d13bf01d4 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:59.177161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:59.178828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:59.238692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 902/1826] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:59 2022] PUT /identity/v3/projects/3a265ce8247f4c3aac708aff32f44df9/users/4cc9d272d6714ad7aac0e208a8ed1175/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:59.240193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fe53a-d7ee-4036-b76e-2893b3a40652 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:59.241393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-de1fe53a-d7ee-4036-b76e-2893b3a40652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:59.255472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefc716-15f7-4adb-b6fb-e99f11a7bc46 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:59.310226 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fe53a-d7ee-4036-b76e-2893b3a40652 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:59.313010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522dcb2-ee8f-438d-bea7-bc7b5efc655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:59.313740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522dcb2-ee8f-438d-bea7-bc7b5efc655c admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:59.314015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522dcb2-ee8f-438d-bea7-bc7b5efc655c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:59.314289 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522dcb2-ee8f-438d-bea7-bc7b5efc655c admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:59.314952 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163add0-fd66-4a6e-a07c-fa17aa1486b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:59.315122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522dcb2-ee8f-438d-bea7-bc7b5efc655c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:59.316073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163add0-fd66-4a6e-a07c-fa17aa1486b9 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:59.316360 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163add0-fd66-4a6e-a07c-fa17aa1486b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:59.316538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522dcb2-ee8f-438d-bea7-bc7b5efc655c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:59.316641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163add0-fd66-4a6e-a07c-fa17aa1486b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:59.317267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163add0-fd66-4a6e-a07c-fa17aa1486b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:56:59.317576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163add0-fd66-4a6e-a07c-fa17aa1486b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:59.319203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163add0-fd66-4a6e-a07c-fa17aa1486b9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:59.326864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 925/1827] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:56:59.328336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c762a7-5ee0-4491-ade2-18e4603eeed3 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:59.328605 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c762a7-5ee0-4491-ade2-18e4603eeed3 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:59.329004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c762a7-5ee0-4491-ade2-18e4603eeed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:59.336298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 903/1828] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:56:59.337548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e241b2a-a1b3-48a5-80fe-e8599607fdbc None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:59.337838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e241b2a-a1b3-48a5-80fe-e8599607fdbc None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:59.338113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e241b2a-a1b3-48a5-80fe-e8599607fdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:59.351142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-38c762a7-5ee0-4491-ade2-18e4603eeed3 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:59.352781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-38c762a7-5ee0-4491-ade2-18e4603eeed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:59.360070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 926/1829] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:59.361123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c762a7-5ee0-4491-ade2-18e4603eeed3 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:59.367759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-9e241b2a-a1b3-48a5-80fe-e8599607fdbc None None] MFA Rules not processed for user `4cc9d272d6714ad7aac0e208a8ed1175`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:59.368936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9e241b2a-a1b3-48a5-80fe-e8599607fdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:59.371517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c762a7-5ee0-4491-ade2-18e4603eeed3 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:59.376223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc95f89-aa19-4909-bf75-0a1b2c856aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:59.379611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc95f89-aa19-4909-bf75-0a1b2c856aff admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:59.379611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc95f89-aa19-4909-bf75-0a1b2c856aff admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:59.379611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc95f89-aa19-4909-bf75-0a1b2c856aff admin admin] PATH_INFO: `/v3/projects/8556d6face314983b0a162018c116d63/users/c1c8801d2d9246c2840cea98975184c8/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:59.379611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc95f89-aa19-4909-bf75-0a1b2c856aff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8556d6face314983b0a162018c116d63, user_id=c1c8801d2d9246c2840cea98975184c8, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:59.382197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc95f89-aa19-4909-bf75-0a1b2c856aff admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:59.392284 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 927/1830] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:56:59 2022] PUT /identity/v3/projects/8556d6face314983b0a162018c116d63/users/c1c8801d2d9246c2840cea98975184c8/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:56:59.393718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc95f89-aa19-4909-bf75-0a1b2c856aff admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:59.443444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 904/1831] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:59.444551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e241b2a-a1b3-48a5-80fe-e8599607fdbc None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:59.446966 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9e241b2a-a1b3-48a5-80fe-e8599607fdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:59.467467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc95f89-aa19-4909-bf75-0a1b2c856aff admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:59.472719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaa347-e95b-4de9-b431-1b0134b41c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:59.473325 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaa347-e95b-4de9-b431-1b0134b41c01 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:59.473600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaa347-e95b-4de9-b431-1b0134b41c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:59.473870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaa347-e95b-4de9-b431-1b0134b41c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:59.474563 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdaa347-e95b-4de9-b431-1b0134b41c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:59.476481 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdaa347-e95b-4de9-b431-1b0134b41c01 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:59.504142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:59.506773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e241b2a-a1b3-48a5-80fe-e8599607fdbc None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:59.560795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 928/1832] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:59 2022] POST /identity/v3/projects => generated 369 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:59.565054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86c7a2-c2a5-4865-95a3-ba73aabc11ea None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:59.566787 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59404749-48fd-43bb-a054-24f6c1168e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:59.567083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86c7a2-c2a5-4865-95a3-ba73aabc11ea None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:59.567599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86c7a2-c2a5-4865-95a3-ba73aabc11ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:59.567925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59404749-48fd-43bb-a054-24f6c1168e05 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:59.568359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59404749-48fd-43bb-a054-24f6c1168e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:59.568762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59404749-48fd-43bb-a054-24f6c1168e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:59.591965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59404749-48fd-43bb-a054-24f6c1168e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:59.593433 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59404749-48fd-43bb-a054-24f6c1168e05 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:59.605845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 905/1833] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:56:59.606836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-8e86c7a2-c2a5-4865-95a3-ba73aabc11ea None None] MFA Rules not processed for user `c1c8801d2d9246c2840cea98975184c8`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:56:59.607746 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59404749-48fd-43bb-a054-24f6c1168e05 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:59.609060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8e86c7a2-c2a5-4865-95a3-ba73aabc11ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:59.614830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-59404749-48fd-43bb-a054-24f6c1168e05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:59.632668 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59404749-48fd-43bb-a054-24f6c1168e05 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:59.722305 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757617e-6998-4df2-b3b7-e2748b78ec87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:59.723102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757617e-6998-4df2-b3b7-e2748b78ec87 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:59.723590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757617e-6998-4df2-b3b7-e2748b78ec87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:59.727516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757617e-6998-4df2-b3b7-e2748b78ec87 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:59.732136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757617e-6998-4df2-b3b7-e2748b78ec87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:59.732136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757617e-6998-4df2-b3b7-e2748b78ec87 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:59.754237 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 929/1834] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:56:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:56:59.755144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86c7a2-c2a5-4865-95a3-ba73aabc11ea None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:59.769588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86c7a2-c2a5-4865-95a3-ba73aabc11ea None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:56:59.775707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffb225-813a-4045-8a8f-c4c0ca40f013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:56:59.776427 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffb225-813a-4045-8a8f-c4c0ca40f013 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:56:59.778650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffb225-813a-4045-8a8f-c4c0ca40f013 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:56:59.778650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffb225-813a-4045-8a8f-c4c0ca40f013 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:56:59.778650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffb225-813a-4045-8a8f-c4c0ca40f013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:56:59.781420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffb225-813a-4045-8a8f-c4c0ca40f013 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:56:59.825274 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:56:59.888960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 906/1835] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:56:59 2022] POST /identity/v3/projects => generated 379 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:59.889425 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757617e-6998-4df2-b3b7-e2748b78ec87 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:59.891342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c757617e-6998-4df2-b3b7-e2748b78ec87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:56:59.925643 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 930/1836] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:59 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:56:59.926935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffb225-813a-4045-8a8f-c4c0ca40f013 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:56:59.946091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffb225-813a-4045-8a8f-c4c0ca40f013 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:00.025392 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757617e-6998-4df2-b3b7-e2748b78ec87 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:00.029520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe121de-f1b4-45ec-b18b-d8e7cbb8570a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:00.030240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe121de-f1b4-45ec-b18b-d8e7cbb8570a service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:00.030787 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe121de-f1b4-45ec-b18b-d8e7cbb8570a service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:00.031143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe121de-f1b4-45ec-b18b-d8e7cbb8570a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:00.056639 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815573c-3c6b-4088-83cf-4411d19290d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:00.060953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815573c-3c6b-4088-83cf-4411d19290d6 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:00.060953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815573c-3c6b-4088-83cf-4411d19290d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:00.060953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815573c-3c6b-4088-83cf-4411d19290d6 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:00.060953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815573c-3c6b-4088-83cf-4411d19290d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:00.060953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815573c-3c6b-4088-83cf-4411d19290d6 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:00.182152 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe121de-f1b4-45ec-b18b-d8e7cbb8570a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:00.183799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe121de-f1b4-45ec-b18b-d8e7cbb8570a service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:00.196889 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 907/1837] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:56:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:00.198124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe121de-f1b4-45ec-b18b-d8e7cbb8570a service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:00.229038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe121de-f1b4-45ec-b18b-d8e7cbb8570a service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:00.238616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 931/1838] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:56:59 2022] POST /identity/v3/users => generated 296 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:00.239945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815573c-3c6b-4088-83cf-4411d19290d6 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:00.243382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee6e65-b329-4f73-8c5f-07beae2b8507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:00.243819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee6e65-b329-4f73-8c5f-07beae2b8507 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:00.244198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee6e65-b329-4f73-8c5f-07beae2b8507 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:00.244598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee6e65-b329-4f73-8c5f-07beae2b8507 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:00.245345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee6e65-b329-4f73-8c5f-07beae2b8507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:00.247567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee6e65-b329-4f73-8c5f-07beae2b8507 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:00.293386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 908/1839] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:00.299110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee6e65-b329-4f73-8c5f-07beae2b8507 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:00.304229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815573c-3c6b-4088-83cf-4411d19290d6 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:00.320578 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de71024-42b7-4097-8d76-7fd4579fd1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:00.320837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee6e65-b329-4f73-8c5f-07beae2b8507 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:00.321385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de71024-42b7-4097-8d76-7fd4579fd1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:00.321633 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de71024-42b7-4097-8d76-7fd4579fd1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:00.321909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de71024-42b7-4097-8d76-7fd4579fd1d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:00.324254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de71024-42b7-4097-8d76-7fd4579fd1d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:00.331799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de71024-42b7-4097-8d76-7fd4579fd1d4 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:00.335230 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395eb1a-5ec9-4b5f-8367-3605c4d7890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:00.335230 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395eb1a-5ec9-4b5f-8367-3605c4d7890f admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:00.339737 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395eb1a-5ec9-4b5f-8367-3605c4d7890f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:00.339737 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395eb1a-5ec9-4b5f-8367-3605c4d7890f admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:00.339737 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395eb1a-5ec9-4b5f-8367-3605c4d7890f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:00.356403 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395eb1a-5ec9-4b5f-8367-3605c4d7890f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:00.365704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 932/1840] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:00.372990 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 909/1841] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:00.375532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395eb1a-5ec9-4b5f-8367-3605c4d7890f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:00.375651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de71024-42b7-4097-8d76-7fd4579fd1d4 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:00.388036 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395eb1a-5ec9-4b5f-8367-3605c4d7890f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:00.388763 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de71024-42b7-4097-8d76-7fd4579fd1d4 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:00.394662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff577848-3c37-4094-8090-46721003aeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:00.395669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff577848-3c37-4094-8090-46721003aeb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:00.395954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff577848-3c37-4094-8090-46721003aeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:00.396227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff577848-3c37-4094-8090-46721003aeb4 admin admin] PATH_INFO: `/v3/projects/8e42a6903eea49faaf9ee72f0de0505c/users/e6887f447d084fb5a7f37378474142cc/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:00.396447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db56f9-0357-498d-bf33-ccf14f6e044d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:00.397400 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db56f9-0357-498d-bf33-ccf14f6e044d admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:00.397752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db56f9-0357-498d-bf33-ccf14f6e044d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:00.398093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db56f9-0357-498d-bf33-ccf14f6e044d admin admin] PATH_INFO: `/v3/projects/ffbfbad479554ac0b415970f9bcf8d11/users/6f8e2c9e7ad042ffb7af6f845d02ebd5/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:00.413361 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff577848-3c37-4094-8090-46721003aeb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e42a6903eea49faaf9ee72f0de0505c, user_id=e6887f447d084fb5a7f37378474142cc, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:00.422872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff577848-3c37-4094-8090-46721003aeb4 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:00.424836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db56f9-0357-498d-bf33-ccf14f6e044d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffbfbad479554ac0b415970f9bcf8d11, user_id=6f8e2c9e7ad042ffb7af6f845d02ebd5, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:00.427885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db56f9-0357-498d-bf33-ccf14f6e044d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:00.541667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 933/1842] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:00 2022] PUT /identity/v3/projects/8e42a6903eea49faaf9ee72f0de0505c/users/e6887f447d084fb5a7f37378474142cc/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:00.552381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e984c34-36b4-44ce-a084-a6663c56815f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:00.552381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e984c34-36b4-44ce-a084-a6663c56815f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:00.552381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e984c34-36b4-44ce-a084-a6663c56815f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:00.593422 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0e984c34-36b4-44ce-a084-a6663c56815f None None] MFA Rules not processed for user `e6887f447d084fb5a7f37378474142cc`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:00.595162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0e984c34-36b4-44ce-a084-a6663c56815f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:00.604808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 910/1843] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:00 2022] PUT /identity/v3/projects/ffbfbad479554ac0b415970f9bcf8d11/users/6f8e2c9e7ad042ffb7af6f845d02ebd5/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:00.609565 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db56f9-0357-498d-bf33-ccf14f6e044d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:00.656441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:00.706868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 934/1844] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:00.712475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e984c34-36b4-44ce-a084-a6663c56815f None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:00.729490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db56f9-0357-498d-bf33-ccf14f6e044d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:00.735488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4bc0e9-2092-4b08-91f4-99757625cb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:00.736518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bc0e9-2092-4b08-91f4-99757625cb29 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:00.736989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bc0e9-2092-4b08-91f4-99757625cb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:00.737262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bc0e9-2092-4b08-91f4-99757625cb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:00.738719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4bc0e9-2092-4b08-91f4-99757625cb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:00.740526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4bc0e9-2092-4b08-91f4-99757625cb29 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:00.772905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 911/1845] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:00.776049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4bc0e9-2092-4b08-91f4-99757625cb29 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:00.800403 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4bc0e9-2092-4b08-91f4-99757625cb29 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:00.823118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e984c34-36b4-44ce-a084-a6663c56815f None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:00.825564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5827079-4937-40a6-bcb3-5cb59130c5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:00.826429 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5827079-4937-40a6-bcb3-5cb59130c5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:00.826743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5827079-4937-40a6-bcb3-5cb59130c5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:00.827024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5827079-4937-40a6-bcb3-5cb59130c5b7 admin admin] PATH_INFO: `/v3/projects/ffbfbad479554ac0b415970f9bcf8d11/users/6f8e2c9e7ad042ffb7af6f845d02ebd5/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:00.829653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e3a03-fd18-4c03-bd38-61fae9c4d5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:00.831145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e3a03-fd18-4c03-bd38-61fae9c4d5ff admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:00.831550 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e3a03-fd18-4c03-bd38-61fae9c4d5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:00.831908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5827079-4937-40a6-bcb3-5cb59130c5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffbfbad479554ac0b415970f9bcf8d11, user_id=6f8e2c9e7ad042ffb7af6f845d02ebd5, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:00.832033 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e3a03-fd18-4c03-bd38-61fae9c4d5ff admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:00.833094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e3a03-fd18-4c03-bd38-61fae9c4d5ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:00.834721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5827079-4937-40a6-bcb3-5cb59130c5b7 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:00.835399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e3a03-fd18-4c03-bd38-61fae9c4d5ff admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:00.850139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 912/1846] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:00 2022] PUT /identity/v3/projects/ffbfbad479554ac0b415970f9bcf8d11/users/6f8e2c9e7ad042ffb7af6f845d02ebd5/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:00.860565 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af486213-6fde-4f7e-9717-332c1e7dfee7 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:00.860847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af486213-6fde-4f7e-9717-332c1e7dfee7 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:00.861243 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af486213-6fde-4f7e-9717-332c1e7dfee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:00.896338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-af486213-6fde-4f7e-9717-332c1e7dfee7 None None] MFA Rules not processed for user `6f8e2c9e7ad042ffb7af6f845d02ebd5`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:00.896338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-af486213-6fde-4f7e-9717-332c1e7dfee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:00.940909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 935/1847] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:00 2022] POST /identity/v3/projects => generated 367 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:00.940909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e3a03-fd18-4c03-bd38-61fae9c4d5ff admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:00.947703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-378e3a03-fd18-4c03-bd38-61fae9c4d5ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:01.072676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 913/1848] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:01.072676 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af486213-6fde-4f7e-9717-332c1e7dfee7 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:01.148519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e3a03-fd18-4c03-bd38-61fae9c4d5ff admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:01.153431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:01.154089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:01.154377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:01.154691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] PATH_INFO: `/v3/projects/e05ad8bb6540472b9eac5e3cf8c4fa45` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:01.155882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e05ad8bb6540472b9eac5e3cf8c4fa45)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:01.158213 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:01.271896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af486213-6fde-4f7e-9717-332c1e7dfee7 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:01.277552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50addd-7114-4888-9380-127765ed1208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:01.278780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50addd-7114-4888-9380-127765ed1208 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:01.279236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50addd-7114-4888-9380-127765ed1208 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:01.279789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50addd-7114-4888-9380-127765ed1208 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:01.280727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50addd-7114-4888-9380-127765ed1208 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:01.283423 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50addd-7114-4888-9380-127765ed1208 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:01.508702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 914/1849] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:01 2022] POST /identity/v3/users => generated 290 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:01.513316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e05ad8bb6540472b9eac5e3cf8c4fa45', 'request_id': 'req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:01.514664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50addd-7114-4888-9380-127765ed1208 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:01.538793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-da50addd-7114-4888-9380-127765ed1208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:01.538897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e05ad8bb6540472b9eac5e3cf8c4fa45', 'request_id': 'req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:01.622357 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50addd-7114-4888-9380-127765ed1208 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:01.648386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e05ad8bb6540472b9eac5e3cf8c4fa45', 'request_id': 'req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:01.668260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] The token cache is being invalidate because project e05ad8bb6540472b9eac5e3cf8c4fa45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:01.669192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:01.669992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:01.672053 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 936/1850] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:00 2022] DELETE /identity/v3/projects/e05ad8bb6540472b9eac5e3cf8c4fa45 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:01.676047 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:01.679823 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:01.687863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504046c-a99e-449d-b077-520b24aa023b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:01.687863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504046c-a99e-449d-b077-520b24aa023b service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:01.687863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504046c-a99e-449d-b077-520b24aa023b service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:01.687863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504046c-a99e-449d-b077-520b24aa023b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:01.689973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9504046c-a99e-449d-b077-520b24aa023b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:01.768583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9504046c-a99e-449d-b077-520b24aa023b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:01.771532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9504046c-a99e-449d-b077-520b24aa023b service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:01.777540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2c890-70da-4baf-8f4f-a5e3ef7da8e1 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:01.783139 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa33d3-4301-4e78-a8f5-05164f7a4ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:01.783314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 915/1851] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:01.784642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa33d3-4301-4e78-a8f5-05164f7a4ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:01.784934 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa33d3-4301-4e78-a8f5-05164f7a4ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:01.785059 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504046c-a99e-449d-b077-520b24aa023b service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:01.785281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa33d3-4301-4e78-a8f5-05164f7a4ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:01.786065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa33d3-4301-4e78-a8f5-05164f7a4ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:01.786382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9504046c-a99e-449d-b077-520b24aa023b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:01.787707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa33d3-4301-4e78-a8f5-05164f7a4ea1 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:01.804781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 937/1852] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:01.805004 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504046c-a99e-449d-b077-520b24aa023b service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:01.806005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa33d3-4301-4e78-a8f5-05164f7a4ea1 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:01.810165 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7571d-1f08-461c-91c6-0ffa689ab69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:01.811084 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7571d-1f08-461c-91c6-0ffa689ab69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:01.811976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7571d-1f08-461c-91c6-0ffa689ab69a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:01.812635 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7571d-1f08-461c-91c6-0ffa689ab69a admin admin] PATH_INFO: `/v3/users/77f8b9fbd65d46c596fe2d430e171f65` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:01.814024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7571d-1f08-461c-91c6-0ffa689ab69a admin admin] RBAC: Authorizing `identity:delete_user(user_id=77f8b9fbd65d46c596fe2d430e171f65)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:01.817321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7571d-1f08-461c-91c6-0ffa689ab69a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:01.818448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa33d3-4301-4e78-a8f5-05164f7a4ea1 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:01.828555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbb532-8383-435e-bc95-359f25b60d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:01.828555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbb532-8383-435e-bc95-359f25b60d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:01.828555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbb532-8383-435e-bc95-359f25b60d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:01.828555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbb532-8383-435e-bc95-359f25b60d2b admin admin] PATH_INFO: `/v3/projects/4a272e111c1c4c96beaa4620313d0f39/users/ca61bbfdb5804317b12080b40275cb18/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:01.852947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbb532-8383-435e-bc95-359f25b60d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a272e111c1c4c96beaa4620313d0f39, user_id=ca61bbfdb5804317b12080b40275cb18, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:01.855057 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbb532-8383-435e-bc95-359f25b60d2b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:01.967243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 938/1853] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:01 2022] PUT /identity/v3/projects/4a272e111c1c4c96beaa4620313d0f39/users/ca61bbfdb5804317b12080b40275cb18/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:01.976727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbb532-8383-435e-bc95-359f25b60d2b admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:02.065260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbb532-8383-435e-bc95-359f25b60d2b admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:02.070387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4af97-0ca0-49a8-aec5-4000130aa005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:02.071139 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4af97-0ca0-49a8-aec5-4000130aa005 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:02.071489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4af97-0ca0-49a8-aec5-4000130aa005 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:02.071830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4af97-0ca0-49a8-aec5-4000130aa005 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:02.072554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4af97-0ca0-49a8-aec5-4000130aa005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:02.074224 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4af97-0ca0-49a8-aec5-4000130aa005 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:02.085708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 939/1854] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:02.089929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4af97-0ca0-49a8-aec5-4000130aa005 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:02.107156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:02.109724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4af97-0ca0-49a8-aec5-4000130aa005 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:02.115609 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641dbca7-e8a7-41f0-97a8-5265980bfdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:02.116282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641dbca7-e8a7-41f0-97a8-5265980bfdbb admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:02.116563 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641dbca7-e8a7-41f0-97a8-5265980bfdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:02.117232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641dbca7-e8a7-41f0-97a8-5265980bfdbb admin admin] PATH_INFO: `/v3/projects/4a272e111c1c4c96beaa4620313d0f39/users/ca61bbfdb5804317b12080b40275cb18/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:02.119330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641dbca7-e8a7-41f0-97a8-5265980bfdbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a272e111c1c4c96beaa4620313d0f39, user_id=ca61bbfdb5804317b12080b40275cb18, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:02.121979 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641dbca7-e8a7-41f0-97a8-5265980bfdbb admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:02.150627 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 940/1855] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:02 2022] PUT /identity/v3/projects/4a272e111c1c4c96beaa4620313d0f39/users/ca61bbfdb5804317b12080b40275cb18/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:02.150627 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641dbca7-e8a7-41f0-97a8-5265980bfdbb admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:02.150627 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-641dbca7-e8a7-41f0-97a8-5265980bfdbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:02.166386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5da7571d-1f08-461c-91c6-0ffa689ab69a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77f8b9fbd65d46c596fe2d430e171f65'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:02.244089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641dbca7-e8a7-41f0-97a8-5265980bfdbb admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:02.248879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6f244-b368-4a78-840a-5c4860457f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:02.252167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6f244-b368-4a78-840a-5c4860457f32 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:02.252167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6f244-b368-4a78-840a-5c4860457f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:02.252167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6f244-b368-4a78-840a-5c4860457f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:02.253090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-eee6f244-b368-4a78-840a-5c4860457f32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:02.270681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5da7571d-1f08-461c-91c6-0ffa689ab69a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77f8b9fbd65d46c596fe2d430e171f65'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:02.274995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5da7571d-1f08-461c-91c6-0ffa689ab69a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77f8b9fbd65d46c596fe2d430e171f65'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:02.274995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5da7571d-1f08-461c-91c6-0ffa689ab69a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77f8b9fbd65d46c596fe2d430e171f65'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:02.281047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5da7571d-1f08-461c-91c6-0ffa689ab69a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77f8b9fbd65d46c596fe2d430e171f65'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:02.303612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 916/1856] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:01 2022] DELETE /identity/v3/users/77f8b9fbd65d46c596fe2d430e171f65 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:02.309456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bbb95-bdf5-4fb4-9f8d-6b5ac9d61a70 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:02.309456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bbb95-bdf5-4fb4-9f8d-6b5ac9d61a70 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:02.309456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bbb95-bdf5-4fb4-9f8d-6b5ac9d61a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:02.325601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6f244-b368-4a78-840a-5c4860457f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:02.327873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6f244-b368-4a78-840a-5c4860457f32 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:02.340548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-dd7bbb95-bdf5-4fb4-9f8d-6b5ac9d61a70 None None] MFA Rules not processed for user `ca61bbfdb5804317b12080b40275cb18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:02.341803 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-dd7bbb95-bdf5-4fb4-9f8d-6b5ac9d61a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:02.363863 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 941/1857] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:02.492054 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 917/1858] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:02.576579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6f244-b368-4a78-840a-5c4860457f32 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:02.578319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-eee6f244-b368-4a78-840a-5c4860457f32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:02.594340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6f244-b368-4a78-840a-5c4860457f32 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:02.687505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c4d9b-ba59-408a-85c7-ce805ff5a461 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:02.689305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c4d9b-ba59-408a-85c7-ce805ff5a461 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:02.690029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c4d9b-ba59-408a-85c7-ce805ff5a461 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:02.690384 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c4d9b-ba59-408a-85c7-ce805ff5a461 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:02.715782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c4d9b-ba59-408a-85c7-ce805ff5a461 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:02.720432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c4d9b-ba59-408a-85c7-ce805ff5a461 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:02.742706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 942/1859] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:02.905918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:02.916769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bbb95-bdf5-4fb4-9f8d-6b5ac9d61a70 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:02.918419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-dd7bbb95-bdf5-4fb4-9f8d-6b5ac9d61a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:02.934287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bbb95-bdf5-4fb4-9f8d-6b5ac9d61a70 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:02.940837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1faebde-bc40-40ff-a901-8cef4ead4112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:02.940837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faebde-bc40-40ff-a901-8cef4ead4112 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:02.940837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faebde-bc40-40ff-a901-8cef4ead4112 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:02.940837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faebde-bc40-40ff-a901-8cef4ead4112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:02.943125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d1faebde-bc40-40ff-a901-8cef4ead4112 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:03.025790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1faebde-bc40-40ff-a901-8cef4ead4112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:03.027604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1faebde-bc40-40ff-a901-8cef4ead4112 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:03.054931 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 918/1860] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:03.680913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c4d9b-ba59-408a-85c7-ce805ff5a461 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:03.691943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-672c4d9b-ba59-408a-85c7-ce805ff5a461 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:03.791813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c4d9b-ba59-408a-85c7-ce805ff5a461 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:03.799755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:03.800612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:03.801207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:03.801720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8 admin admin] PATH_INFO: `/v3/projects/6c238134c7ac460da94da62a78c406d7` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:03.803252 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c238134c7ac460da94da62a78c406d7)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:03.806652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:03.974725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c238134c7ac460da94da62a78c406d7', 'request_id': 'req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:03.982035 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c238134c7ac460da94da62a78c406d7', 'request_id': 'req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:04.048386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:04.057033 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c238134c7ac460da94da62a78c406d7', 'request_id': 'req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:04.071068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8 admin admin] The token cache is being invalidate because project 6c238134c7ac460da94da62a78c406d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:04.071477 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:04.072211 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-fab55fc6-23fa-4a49-bbbc-d6d7cb03e6b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:04.074030 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 943/1861] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:03 2022] DELETE /identity/v3/projects/6c238134c7ac460da94da62a78c406d7 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:04.079344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6898026-af93-48e4-919b-959e76bc5f06 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:04.079845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6898026-af93-48e4-919b-959e76bc5f06 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:04.079928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6898026-af93-48e4-919b-959e76bc5f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:04.122597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-d6898026-af93-48e4-919b-959e76bc5f06 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:04.122597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d6898026-af93-48e4-919b-959e76bc5f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:04.263213 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 919/1862] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:04.264560 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb641059-dec5-48c6-bef2-4390ed025142 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:04.264873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb641059-dec5-48c6-bef2-4390ed025142 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:04.265150 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb641059-dec5-48c6-bef2-4390ed025142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:04.293598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-fb641059-dec5-48c6-bef2-4390ed025142 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:04.296331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fb641059-dec5-48c6-bef2-4390ed025142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:04.316832 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 944/1863] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:04.321279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6898026-af93-48e4-919b-959e76bc5f06 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:04.335869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6898026-af93-48e4-919b-959e76bc5f06 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:04.375842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d4d7a-c788-49de-9559-f1b5bd4821fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:04.376334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d4d7a-c788-49de-9559-f1b5bd4821fd admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:04.376619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d4d7a-c788-49de-9559-f1b5bd4821fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:04.376893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d4d7a-c788-49de-9559-f1b5bd4821fd admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:04.378428 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d4d7a-c788-49de-9559-f1b5bd4821fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:04.378775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d4d7a-c788-49de-9559-f1b5bd4821fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:04.381597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d4d7a-c788-49de-9559-f1b5bd4821fd admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:04.428769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 920/1864] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:04.433030 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb641059-dec5-48c6-bef2-4390ed025142 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:04.448625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb641059-dec5-48c6-bef2-4390ed025142 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:04.456070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600ab11-cfd4-4ed5-a48b-d4f6ec172d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:04.458160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600ab11-cfd4-4ed5-a48b-d4f6ec172d4e admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:04.461558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600ab11-cfd4-4ed5-a48b-d4f6ec172d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:04.461558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600ab11-cfd4-4ed5-a48b-d4f6ec172d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:04.461558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600ab11-cfd4-4ed5-a48b-d4f6ec172d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:04.463415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600ab11-cfd4-4ed5-a48b-d4f6ec172d4e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:04.549416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 945/1865] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:04 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:04.554403 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d4d7a-c788-49de-9559-f1b5bd4821fd admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:04.566322 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d4d7a-c788-49de-9559-f1b5bd4821fd admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:04.655750 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5300314-f457-4db0-91ca-574d961d6e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:04.656437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5300314-f457-4db0-91ca-574d961d6e9f admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:04.656769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5300314-f457-4db0-91ca-574d961d6e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:04.657089 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5300314-f457-4db0-91ca-574d961d6e9f admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:04.657821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5300314-f457-4db0-91ca-574d961d6e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:04.660050 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5300314-f457-4db0-91ca-574d961d6e9f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:04.763157 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 921/1866] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:04 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:04.767122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600ab11-cfd4-4ed5-a48b-d4f6ec172d4e admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:04.786830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600ab11-cfd4-4ed5-a48b-d4f6ec172d4e admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:04.795539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86f65f-244d-4f18-b1d5-a41d47c5894c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:04.797846 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86f65f-244d-4f18-b1d5-a41d47c5894c admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:04.798272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86f65f-244d-4f18-b1d5-a41d47c5894c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:04.798996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86f65f-244d-4f18-b1d5-a41d47c5894c admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:04.800972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86f65f-244d-4f18-b1d5-a41d47c5894c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:04.803129 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86f65f-244d-4f18-b1d5-a41d47c5894c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:04.821789 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 946/1867] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:04.826125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5300314-f457-4db0-91ca-574d961d6e9f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:04.840055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5300314-f457-4db0-91ca-574d961d6e9f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:04.846453 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee6bc0-b82e-45c2-92dc-c7b0d6c05e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:04.847077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee6bc0-b82e-45c2-92dc-c7b0d6c05e83 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:04.847361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee6bc0-b82e-45c2-92dc-c7b0d6c05e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:04.847637 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee6bc0-b82e-45c2-92dc-c7b0d6c05e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:04.848533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee6bc0-b82e-45c2-92dc-c7b0d6c05e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:04.850258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee6bc0-b82e-45c2-92dc-c7b0d6c05e83 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:04.865402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 922/1868] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:04.878581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86f65f-244d-4f18-b1d5-a41d47c5894c admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:04.902427 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86f65f-244d-4f18-b1d5-a41d47c5894c admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:04.907366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad6af7-a4d4-48d5-817c-9843b195a906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:04.908031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad6af7-a4d4-48d5-817c-9843b195a906 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:04.908314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad6af7-a4d4-48d5-817c-9843b195a906 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:04.908712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad6af7-a4d4-48d5-817c-9843b195a906 admin admin] PATH_INFO: `/v3/projects/49efc288d62f4ebb8173c59c3885f2fc/users/89e2d95af5ca4a659aac274b4f3568af/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:04.928574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad6af7-a4d4-48d5-817c-9843b195a906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49efc288d62f4ebb8173c59c3885f2fc, user_id=89e2d95af5ca4a659aac274b4f3568af, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:04.934846 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad6af7-a4d4-48d5-817c-9843b195a906 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:05.008610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 947/1869] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:04 2022] PUT /identity/v3/projects/49efc288d62f4ebb8173c59c3885f2fc/users/89e2d95af5ca4a659aac274b4f3568af/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:05.022872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29350312-d1b3-498d-8d62-f0e05c8ac41f None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:05.023547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29350312-d1b3-498d-8d62-f0e05c8ac41f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:05.024360 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29350312-d1b3-498d-8d62-f0e05c8ac41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:05.063520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-29350312-d1b3-498d-8d62-f0e05c8ac41f None None] MFA Rules not processed for user `89e2d95af5ca4a659aac274b4f3568af`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:05.067844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-29350312-d1b3-498d-8d62-f0e05c8ac41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:05.149543 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 923/1870] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:05.154529 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad6af7-a4d4-48d5-817c-9843b195a906 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:05.223374 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:05.230587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad6af7-a4d4-48d5-817c-9843b195a906 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:05.236496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1aa9f-810d-4c87-8716-24fb1a229483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:05.237515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1aa9f-810d-4c87-8716-24fb1a229483 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:05.237994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1aa9f-810d-4c87-8716-24fb1a229483 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:05.238336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1aa9f-810d-4c87-8716-24fb1a229483 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:05.239183 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1aa9f-810d-4c87-8716-24fb1a229483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:05.241891 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1aa9f-810d-4c87-8716-24fb1a229483 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:05.278813 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29350312-d1b3-498d-8d62-f0e05c8ac41f None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:05.280884 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-29350312-d1b3-498d-8d62-f0e05c8ac41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:05.303317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29350312-d1b3-498d-8d62-f0e05c8ac41f None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:05.309027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669a8c3-08c0-4c16-aea3-7305e110ef48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:05.309679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669a8c3-08c0-4c16-aea3-7305e110ef48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:05.310102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669a8c3-08c0-4c16-aea3-7305e110ef48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:05.310386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669a8c3-08c0-4c16-aea3-7305e110ef48 admin admin] PATH_INFO: `/v3/users/cb75c3a815494c3ea3153d01866f044b` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:05.311855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669a8c3-08c0-4c16-aea3-7305e110ef48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb75c3a815494c3ea3153d01866f044b)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:05.313737 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669a8c3-08c0-4c16-aea3-7305e110ef48 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:05.348146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 948/1871] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:05 2022] POST /identity/v3/projects => generated 387 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:05.352764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1aa9f-810d-4c87-8716-24fb1a229483 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:05.446364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1aa9f-810d-4c87-8716-24fb1a229483 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:05.451347 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aff3e2-03df-464e-8ead-d9f9509f208e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:05.452061 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aff3e2-03df-464e-8ead-d9f9509f208e admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:05.452668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aff3e2-03df-464e-8ead-d9f9509f208e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:05.453004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aff3e2-03df-464e-8ead-d9f9509f208e admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:05.453719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aff3e2-03df-464e-8ead-d9f9509f208e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:05.456359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aff3e2-03df-464e-8ead-d9f9509f208e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:05.562644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 949/1872] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:05 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:57:05.563110 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aff3e2-03df-464e-8ead-d9f9509f208e admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:05.576922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aff3e2-03df-464e-8ead-d9f9509f208e admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:05.589663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5b8a8-6af8-44a1-bc60-996a3589499d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:05.590583 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5b8a8-6af8-44a1-bc60-996a3589499d admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:05.590933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5b8a8-6af8-44a1-bc60-996a3589499d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:05.591274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5b8a8-6af8-44a1-bc60-996a3589499d admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:05.592736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5b8a8-6af8-44a1-bc60-996a3589499d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:05.595432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5b8a8-6af8-44a1-bc60-996a3589499d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:05.614034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 950/1873] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:05.619629 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5b8a8-6af8-44a1-bc60-996a3589499d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:05.643769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5b8a8-6af8-44a1-bc60-996a3589499d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:05.643903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-8669a8c3-08c0-4c16-aea3-7305e110ef48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb75c3a815494c3ea3153d01866f044b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:05.643985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11459ca8-fbec-4023-9eed-349e579f40e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:05.643985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11459ca8-fbec-4023-9eed-349e579f40e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:05.644719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11459ca8-fbec-4023-9eed-349e579f40e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:05.645072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11459ca8-fbec-4023-9eed-349e579f40e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:05.646387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11459ca8-fbec-4023-9eed-349e579f40e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:05.648845 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11459ca8-fbec-4023-9eed-349e579f40e1 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:05.666369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 951/1874] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:05.670599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11459ca8-fbec-4023-9eed-349e579f40e1 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:05.683907 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11459ca8-fbec-4023-9eed-349e579f40e1 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:05.690125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035f4a8-5787-43ee-be95-a7e2bea73887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:05.691535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035f4a8-5787-43ee-be95-a7e2bea73887 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:05.692178 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035f4a8-5787-43ee-be95-a7e2bea73887 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:05.692661 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035f4a8-5787-43ee-be95-a7e2bea73887 admin admin] PATH_INFO: `/v3/projects/2cc9e578e09e465da03688f812b1a8c3/users/e05194319d16404e874a338c8f8ab919/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:05.716451 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035f4a8-5787-43ee-be95-a7e2bea73887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc9e578e09e465da03688f812b1a8c3, user_id=e05194319d16404e874a338c8f8ab919, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:05.720240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035f4a8-5787-43ee-be95-a7e2bea73887 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:05.723476 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-8669a8c3-08c0-4c16-aea3-7305e110ef48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb75c3a815494c3ea3153d01866f044b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:05.723476 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-8669a8c3-08c0-4c16-aea3-7305e110ef48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb75c3a815494c3ea3153d01866f044b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:05.724102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-8669a8c3-08c0-4c16-aea3-7305e110ef48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb75c3a815494c3ea3153d01866f044b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:05.733204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-8669a8c3-08c0-4c16-aea3-7305e110ef48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb75c3a815494c3ea3153d01866f044b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:05.768296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 924/1875] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:05 2022] DELETE /identity/v3/users/cb75c3a815494c3ea3153d01866f044b => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:05.774046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9bbeb-24aa-48f8-b965-c554f56e1968 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:05.774046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9bbeb-24aa-48f8-b965-c554f56e1968 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:05.774281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9bbeb-24aa-48f8-b965-c554f56e1968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:05.797718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 952/1876] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:05 2022] PUT /identity/v3/projects/2cc9e578e09e465da03688f812b1a8c3/users/e05194319d16404e874a338c8f8ab919/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:05.809551 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa44c2d-c3ea-462b-85a3-b7105a41ecea None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:05.810204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa44c2d-c3ea-462b-85a3-b7105a41ecea None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:05.810856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa44c2d-c3ea-462b-85a3-b7105a41ecea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:05.838697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-a9f9bbeb-24aa-48f8-b965-c554f56e1968 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:05.838697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a9f9bbeb-24aa-48f8-b965-c554f56e1968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:05.874027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-faa44c2d-c3ea-462b-85a3-b7105a41ecea None None] MFA Rules not processed for user `e05194319d16404e874a338c8f8ab919`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:05.875459 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-faa44c2d-c3ea-462b-85a3-b7105a41ecea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:05.950449 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 925/1877] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:05.961302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9bbeb-24aa-48f8-b965-c554f56e1968 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:05.962910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a9f9bbeb-24aa-48f8-b965-c554f56e1968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:05.981834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 953/1878] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:05.986234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa44c2d-c3ea-462b-85a3-b7105a41ecea None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:05.988704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-faa44c2d-c3ea-462b-85a3-b7105a41ecea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:06.005049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa44c2d-c3ea-462b-85a3-b7105a41ecea None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:06.009855 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3952f-7d06-4104-90cb-eb7669627c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:06.010466 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3952f-7d06-4104-90cb-eb7669627c8c admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:06.010765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3952f-7d06-4104-90cb-eb7669627c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:06.011036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3952f-7d06-4104-90cb-eb7669627c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:06.012205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3952f-7d06-4104-90cb-eb7669627c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:06.014754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3952f-7d06-4104-90cb-eb7669627c8c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:06.027895 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9bbeb-24aa-48f8-b965-c554f56e1968 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:06.041094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e407939-497c-435a-b555-b76cd5dff01b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:06.041094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e407939-497c-435a-b555-b76cd5dff01b service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:06.041094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e407939-497c-435a-b555-b76cd5dff01b service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:06.041094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e407939-497c-435a-b555-b76cd5dff01b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:06.042925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e407939-497c-435a-b555-b76cd5dff01b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:06.044361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e407939-497c-435a-b555-b76cd5dff01b service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:06.054195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 926/1879] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:57:06.209078 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 954/1880] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:05 2022] POST /identity/v3/projects => generated 387 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:06.214906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e407939-497c-435a-b555-b76cd5dff01b service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:06.242926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e407939-497c-435a-b555-b76cd5dff01b service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:06.318653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979aa87-0bcd-4f24-af42-7ecaef7715f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:06.318653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979aa87-0bcd-4f24-af42-7ecaef7715f1 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:06.318653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979aa87-0bcd-4f24-af42-7ecaef7715f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:06.318653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979aa87-0bcd-4f24-af42-7ecaef7715f1 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:06.319046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979aa87-0bcd-4f24-af42-7ecaef7715f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:06.319046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979aa87-0bcd-4f24-af42-7ecaef7715f1 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:06.422311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:06.461089 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 927/1881] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:06 2022] POST /identity/v3/users => generated 300 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:06.466176 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3952f-7d06-4104-90cb-eb7669627c8c admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:06.478923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3952f-7d06-4104-90cb-eb7669627c8c admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:06.497838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734f7bc-5416-43ac-be42-d83a1ab9c4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:06.498633 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734f7bc-5416-43ac-be42-d83a1ab9c4de admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:06.498982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734f7bc-5416-43ac-be42-d83a1ab9c4de admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:06.499359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734f7bc-5416-43ac-be42-d83a1ab9c4de admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:06.500147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734f7bc-5416-43ac-be42-d83a1ab9c4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:06.501823 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734f7bc-5416-43ac-be42-d83a1ab9c4de admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:06.522714 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 955/1882] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:06.524064 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979aa87-0bcd-4f24-af42-7ecaef7715f1 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:06.535434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979aa87-0bcd-4f24-af42-7ecaef7715f1 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:06.540630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2136df-b718-438c-819b-d0276934e28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:06.541409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2136df-b718-438c-819b-d0276934e28c admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:06.541790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2136df-b718-438c-819b-d0276934e28c admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:06.542164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2136df-b718-438c-819b-d0276934e28c admin admin] PATH_INFO: `/v3/projects/d804ababca4e48039d7daa25b6de4a41/users/cb1971941f8944f88b968d9a78bba42a/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:06.546716 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:06.561692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2136df-b718-438c-819b-d0276934e28c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d804ababca4e48039d7daa25b6de4a41, user_id=cb1971941f8944f88b968d9a78bba42a, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:06.569879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2136df-b718-438c-819b-d0276934e28c admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:06.571512 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734f7bc-5416-43ac-be42-d83a1ab9c4de admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:06.576474 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0734f7bc-5416-43ac-be42-d83a1ab9c4de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:06.591426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734f7bc-5416-43ac-be42-d83a1ab9c4de admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:06.598671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9130bb8-9440-451f-a787-5bcd41789f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:06.599507 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9130bb8-9440-451f-a787-5bcd41789f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:06.599797 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9130bb8-9440-451f-a787-5bcd41789f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:06.600160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9130bb8-9440-451f-a787-5bcd41789f24 admin admin] PATH_INFO: `/v3/projects/80ea9150fd3845a799a5f90bfc893351` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:06.602072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9130bb8-9440-451f-a787-5bcd41789f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ea9150fd3845a799a5f90bfc893351)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:06.610150 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9130bb8-9440-451f-a787-5bcd41789f24 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:06.666382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 928/1883] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:06 2022] PUT /identity/v3/projects/d804ababca4e48039d7daa25b6de4a41/users/cb1971941f8944f88b968d9a78bba42a/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:06.670063 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2136df-b718-438c-819b-d0276934e28c admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:06.746607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2136df-b718-438c-819b-d0276934e28c admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:06.753549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037db22-c899-4f77-87e9-122db7960a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:06.754022 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037db22-c899-4f77-87e9-122db7960a99 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:06.754442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037db22-c899-4f77-87e9-122db7960a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:06.756799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037db22-c899-4f77-87e9-122db7960a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:06.756799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037db22-c899-4f77-87e9-122db7960a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:06.757210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037db22-c899-4f77-87e9-122db7960a99 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:06.772866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 929/1884] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:06.778291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037db22-c899-4f77-87e9-122db7960a99 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:06.782037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f9130bb8-9440-451f-a787-5bcd41789f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ea9150fd3845a799a5f90bfc893351', 'request_id': 'req-f9130bb8-9440-451f-a787-5bcd41789f24'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:06.790332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f9130bb8-9440-451f-a787-5bcd41789f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ea9150fd3845a799a5f90bfc893351', 'request_id': 'req-f9130bb8-9440-451f-a787-5bcd41789f24'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:06.792131 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037db22-c899-4f77-87e9-122db7960a99 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:06.869889 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e875cbe-89b2-4b72-a18d-98b2a9c43c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:06.870594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f9130bb8-9440-451f-a787-5bcd41789f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ea9150fd3845a799a5f90bfc893351', 'request_id': 'req-f9130bb8-9440-451f-a787-5bcd41789f24'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:06.871099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e875cbe-89b2-4b72-a18d-98b2a9c43c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:06.871467 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e875cbe-89b2-4b72-a18d-98b2a9c43c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:06.871674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e875cbe-89b2-4b72-a18d-98b2a9c43c17 admin admin] PATH_INFO: `/v3/projects/d804ababca4e48039d7daa25b6de4a41/users/cb1971941f8944f88b968d9a78bba42a/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:06.873857 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e875cbe-89b2-4b72-a18d-98b2a9c43c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d804ababca4e48039d7daa25b6de4a41, user_id=cb1971941f8944f88b968d9a78bba42a, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:06.876735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e875cbe-89b2-4b72-a18d-98b2a9c43c17 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:06.884217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f9130bb8-9440-451f-a787-5bcd41789f24 admin admin] The token cache is being invalidate because project 80ea9150fd3845a799a5f90bfc893351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:06.884616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f9130bb8-9440-451f-a787-5bcd41789f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:06.885440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f9130bb8-9440-451f-a787-5bcd41789f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:06.890153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 930/1885] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:06 2022] PUT /identity/v3/projects/d804ababca4e48039d7daa25b6de4a41/users/cb1971941f8944f88b968d9a78bba42a/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:06.891972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 956/1886] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:06 2022] DELETE /identity/v3/projects/80ea9150fd3845a799a5f90bfc893351 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:06.897120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e875cbe-89b2-4b72-a18d-98b2a9c43c17 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:06.899113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0e875cbe-89b2-4b72-a18d-98b2a9c43c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:06.902201 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4a65-d513-48f4-b4dc-54599be0e09e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:06.902601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4a65-d513-48f4-b4dc-54599be0e09e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:06.902962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4a65-d513-48f4-b4dc-54599be0e09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:06.934996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-7b9c4a65-d513-48f4-b4dc-54599be0e09e None None] MFA Rules not processed for user `cb1971941f8944f88b968d9a78bba42a`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:06.936610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7b9c4a65-d513-48f4-b4dc-54599be0e09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:06.974224 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e875cbe-89b2-4b72-a18d-98b2a9c43c17 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:06.979930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703dd5d-6d8d-4bb1-b9a2-89e7148ec430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:06.980570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703dd5d-6d8d-4bb1-b9a2-89e7148ec430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:06.980848 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703dd5d-6d8d-4bb1-b9a2-89e7148ec430 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:06.981120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703dd5d-6d8d-4bb1-b9a2-89e7148ec430 admin admin] PATH_INFO: `/v3/users/c1c8801d2d9246c2840cea98975184c8` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:06.982268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703dd5d-6d8d-4bb1-b9a2-89e7148ec430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c8801d2d9246c2840cea98975184c8)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:06.984228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703dd5d-6d8d-4bb1-b9a2-89e7148ec430 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:07.078838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 957/1887] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:07.091105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4a65-d513-48f4-b4dc-54599be0e09e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:07.093842 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7b9c4a65-d513-48f4-b4dc-54599be0e09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:07.183646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4a65-d513-48f4-b4dc-54599be0e09e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:07.189977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8e837-1051-47bc-bcef-5ea459d3b538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:07.189977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8e837-1051-47bc-bcef-5ea459d3b538 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:07.189977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8e837-1051-47bc-bcef-5ea459d3b538 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:07.189977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8e837-1051-47bc-bcef-5ea459d3b538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:07.198723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8e837-1051-47bc-bcef-5ea459d3b538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:07.200146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8e837-1051-47bc-bcef-5ea459d3b538 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:07.200833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c703dd5d-6d8d-4bb1-b9a2-89e7148ec430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c8801d2d9246c2840cea98975184c8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:07.210819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 958/1888] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:57:07.270505 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c703dd5d-6d8d-4bb1-b9a2-89e7148ec430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c8801d2d9246c2840cea98975184c8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:07.271379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c703dd5d-6d8d-4bb1-b9a2-89e7148ec430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c8801d2d9246c2840cea98975184c8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:07.271785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c703dd5d-6d8d-4bb1-b9a2-89e7148ec430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c8801d2d9246c2840cea98975184c8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:07.279551 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c703dd5d-6d8d-4bb1-b9a2-89e7148ec430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c8801d2d9246c2840cea98975184c8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:07.298220 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 931/1889] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:06 2022] DELETE /identity/v3/users/c1c8801d2d9246c2840cea98975184c8 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:07.980898 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8e837-1051-47bc-bcef-5ea459d3b538 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:07.993911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8e837-1051-47bc-bcef-5ea459d3b538 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:08.068943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab14175-d1ea-47b9-9887-bd220bc486a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:08.069539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab14175-d1ea-47b9-9887-bd220bc486a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:08.069813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab14175-d1ea-47b9-9887-bd220bc486a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:08.070082 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab14175-d1ea-47b9-9887-bd220bc486a8 admin admin] PATH_INFO: `/v3/projects/8556d6face314983b0a162018c116d63` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:08.071250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab14175-d1ea-47b9-9887-bd220bc486a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8556d6face314983b0a162018c116d63)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:08.073136 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab14175-d1ea-47b9-9887-bd220bc486a8 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:08.175225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:08.215012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-2ab14175-d1ea-47b9-9887-bd220bc486a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8556d6face314983b0a162018c116d63', 'request_id': 'req-2ab14175-d1ea-47b9-9887-bd220bc486a8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:08.224272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-2ab14175-d1ea-47b9-9887-bd220bc486a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8556d6face314983b0a162018c116d63', 'request_id': 'req-2ab14175-d1ea-47b9-9887-bd220bc486a8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:08.310700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-2ab14175-d1ea-47b9-9887-bd220bc486a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8556d6face314983b0a162018c116d63', 'request_id': 'req-2ab14175-d1ea-47b9-9887-bd220bc486a8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:08.323895 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-2ab14175-d1ea-47b9-9887-bd220bc486a8 admin admin] The token cache is being invalidate because project 8556d6face314983b0a162018c116d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:08.324441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-2ab14175-d1ea-47b9-9887-bd220bc486a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:08.325896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-2ab14175-d1ea-47b9-9887-bd220bc486a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:08.329279 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 959/1890] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:07 2022] DELETE /identity/v3/projects/8556d6face314983b0a162018c116d63 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:08.334444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683dd50-3ffa-48a7-b9fb-a4affe4d4f7e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:08.335007 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683dd50-3ffa-48a7-b9fb-a4affe4d4f7e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:08.335161 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683dd50-3ffa-48a7-b9fb-a4affe4d4f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:08.365467 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-b683dd50-3ffa-48a7-b9fb-a4affe4d4f7e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:08.367135 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b683dd50-3ffa-48a7-b9fb-a4affe4d4f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:08.412782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:08.525086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31018aa5-1ac9-462b-a95a-43b9106f1e47 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:08.525477 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31018aa5-1ac9-462b-a95a-43b9106f1e47 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:08.525877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31018aa5-1ac9-462b-a95a-43b9106f1e47 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:08.536292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 960/1891] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 12:57:08 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:57:08.547331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 932/1892] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:08.558770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44bccb-9a62-4fd6-8eb6-279662bc0b94 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:08.559062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44bccb-9a62-4fd6-8eb6-279662bc0b94 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:08.559240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac26371-c0f8-4d24-a6b3-1aec1307d933 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:08.559348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44bccb-9a62-4fd6-8eb6-279662bc0b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:08.559694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac26371-c0f8-4d24-a6b3-1aec1307d933 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:08.560151 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac26371-c0f8-4d24-a6b3-1aec1307d933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:08.588301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-3f44bccb-9a62-4fd6-8eb6-279662bc0b94 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:08.589297 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3f44bccb-9a62-4fd6-8eb6-279662bc0b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:08.602535 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 933/1893] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:08.604245 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44bccb-9a62-4fd6-8eb6-279662bc0b94 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:08.605722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-bac26371-c0f8-4d24-a6b3-1aec1307d933 None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:08.607696 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-bac26371-c0f8-4d24-a6b3-1aec1307d933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:08.622322 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44bccb-9a62-4fd6-8eb6-279662bc0b94 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:08.628058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e2724-90ee-4fd3-b1af-b13779af6f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:08.642134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e2724-90ee-4fd3-b1af-b13779af6f9a admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:08.642686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e2724-90ee-4fd3-b1af-b13779af6f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:08.643649 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e2724-90ee-4fd3-b1af-b13779af6f9a admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:08.646953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e2724-90ee-4fd3-b1af-b13779af6f9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:08.647707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e2724-90ee-4fd3-b1af-b13779af6f9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:08.653804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e2724-90ee-4fd3-b1af-b13779af6f9a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:08.680878 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 934/1894] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:08.688494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e2724-90ee-4fd3-b1af-b13779af6f9a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:08.704081 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e2724-90ee-4fd3-b1af-b13779af6f9a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:08.710798 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501594e-54c3-4122-a67d-8a5be5ba93f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:08.711589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501594e-54c3-4122-a67d-8a5be5ba93f6 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:08.712031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501594e-54c3-4122-a67d-8a5be5ba93f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:08.712453 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501594e-54c3-4122-a67d-8a5be5ba93f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:08.713408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501594e-54c3-4122-a67d-8a5be5ba93f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:08.715559 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501594e-54c3-4122-a67d-8a5be5ba93f6 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:08.746967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 961/1895] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 12:57:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:08.759103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac26371-c0f8-4d24-a6b3-1aec1307d933 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:08.760680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-bac26371-c0f8-4d24-a6b3-1aec1307d933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:08.826225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac26371-c0f8-4d24-a6b3-1aec1307d933 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:08.830954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519e121-20e5-43c8-85b1-73fed34efff3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:08.831599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519e121-20e5-43c8-85b1-73fed34efff3 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:08.831877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519e121-20e5-43c8-85b1-73fed34efff3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:08.832153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519e121-20e5-43c8-85b1-73fed34efff3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:08.834071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7519e121-20e5-43c8-85b1-73fed34efff3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:08.915160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519e121-20e5-43c8-85b1-73fed34efff3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:08.915687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 935/1896] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:08 2022] POST /identity/v3/projects => generated 379 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:57:08.916692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501594e-54c3-4122-a67d-8a5be5ba93f6 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:08.918323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b501594e-54c3-4122-a67d-8a5be5ba93f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:08.919864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519e121-20e5-43c8-85b1-73fed34efff3 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:08.944034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 962/1897] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:57:08.945355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519e121-20e5-43c8-85b1-73fed34efff3 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:08.968006 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519e121-20e5-43c8-85b1-73fed34efff3 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:09.030770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501594e-54c3-4122-a67d-8a5be5ba93f6 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:09.037128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa34955-b6cf-4870-8801-41695bf94196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:09.037767 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa34955-b6cf-4870-8801-41695bf94196 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:09.038029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa34955-b6cf-4870-8801-41695bf94196 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:09.038436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa34955-b6cf-4870-8801-41695bf94196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:09.048638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa34955-b6cf-4870-8801-41695bf94196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:09.052497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa34955-b6cf-4870-8801-41695bf94196 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:09.062994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 936/1898] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:09.067012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27926a5f-52d1-45cc-a112-93260101d86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:09.067940 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27926a5f-52d1-45cc-a112-93260101d86f admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:09.068752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27926a5f-52d1-45cc-a112-93260101d86f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:09.069051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27926a5f-52d1-45cc-a112-93260101d86f admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:09.069766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27926a5f-52d1-45cc-a112-93260101d86f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:09.071879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27926a5f-52d1-45cc-a112-93260101d86f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:09.151664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 963/1899] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:08 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:09.155660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa34955-b6cf-4870-8801-41695bf94196 service nova] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:09.168042 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa34955-b6cf-4870-8801-41695bf94196 service nova] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:09.172707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aa9ae-48d2-40d6-939f-a4697e4cf4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:09.179701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa9ae-48d2-40d6-939f-a4697e4cf4cf admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:09.179701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa9ae-48d2-40d6-939f-a4697e4cf4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:09.179701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aa9ae-48d2-40d6-939f-a4697e4cf4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:09.179701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816aa9ae-48d2-40d6-939f-a4697e4cf4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:09.179701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816aa9ae-48d2-40d6-939f-a4697e4cf4cf admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:09.186314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 937/1900] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:09.190676 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27926a5f-52d1-45cc-a112-93260101d86f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:09.200409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27926a5f-52d1-45cc-a112-93260101d86f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:09.205694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb00564-29ad-46ce-8e84-c5a72a11068d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:09.206285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb00564-29ad-46ce-8e84-c5a72a11068d admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:09.206614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb00564-29ad-46ce-8e84-c5a72a11068d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:09.214088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb00564-29ad-46ce-8e84-c5a72a11068d admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:09.214088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb00564-29ad-46ce-8e84-c5a72a11068d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:09.214088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb00564-29ad-46ce-8e84-c5a72a11068d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:09.218976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 964/1901] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:09.224622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aa9ae-48d2-40d6-939f-a4697e4cf4cf admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:09.241720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aa9ae-48d2-40d6-939f-a4697e4cf4cf admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:09.250528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28113e-89a4-4d50-898c-8e94eb0dcec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:09.251338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28113e-89a4-4d50-898c-8e94eb0dcec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:09.251761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28113e-89a4-4d50-898c-8e94eb0dcec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:09.252364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28113e-89a4-4d50-898c-8e94eb0dcec2 admin admin] PATH_INFO: `/v3/projects/a71c46f52c924e4cb429335c950d43c4/users/858f72c8c5ff49b6977bae6096bc881b/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:09.268885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28113e-89a4-4d50-898c-8e94eb0dcec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71c46f52c924e4cb429335c950d43c4, user_id=858f72c8c5ff49b6977bae6096bc881b, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:09.272328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28113e-89a4-4d50-898c-8e94eb0dcec2 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:09.407710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 938/1902] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:09 2022] PUT /identity/v3/projects/a71c46f52c924e4cb429335c950d43c4/users/858f72c8c5ff49b6977bae6096bc881b/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:09.418989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe45b-c1a2-4f61-9d0d-489d28118bbf None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:09.419310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe45b-c1a2-4f61-9d0d-489d28118bbf None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:09.419646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe45b-c1a2-4f61-9d0d-489d28118bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:09.455610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-ecdfe45b-c1a2-4f61-9d0d-489d28118bbf None None] MFA Rules not processed for user `858f72c8c5ff49b6977bae6096bc881b`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:09.455610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ecdfe45b-c1a2-4f61-9d0d-489d28118bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:09.549407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 965/1903] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:09.554704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28113e-89a4-4d50-898c-8e94eb0dcec2 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:09.623966 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28113e-89a4-4d50-898c-8e94eb0dcec2 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:09.629311 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2958936-36ae-44cc-ad7a-c4c6c529a32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:09.629907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2958936-36ae-44cc-ad7a-c4c6c529a32c admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:09.630180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2958936-36ae-44cc-ad7a-c4c6c529a32c admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:09.630445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2958936-36ae-44cc-ad7a-c4c6c529a32c admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:09.631109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2958936-36ae-44cc-ad7a-c4c6c529a32c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:09.632967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2958936-36ae-44cc-ad7a-c4c6c529a32c admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:09.669265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:09.713618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfe45b-c1a2-4f61-9d0d-489d28118bbf None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:09.715030 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ecdfe45b-c1a2-4f61-9d0d-489d28118bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:09.728018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfe45b-c1a2-4f61-9d0d-489d28118bbf None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:09.732773 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283b3df-26fc-4a52-8281-214201f2757d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:09.733395 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283b3df-26fc-4a52-8281-214201f2757d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:09.733671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283b3df-26fc-4a52-8281-214201f2757d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:09.733937 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283b3df-26fc-4a52-8281-214201f2757d admin admin] PATH_INFO: `/v3/users/e6887f447d084fb5a7f37378474142cc` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:09.735122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283b3df-26fc-4a52-8281-214201f2757d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6887f447d084fb5a7f37378474142cc)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:09.737013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283b3df-26fc-4a52-8281-214201f2757d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:09.772426 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 939/1904] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:09 2022] POST /identity/v3/projects => generated 379 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:09.775013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2958936-36ae-44cc-ad7a-c4c6c529a32c admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:09.835964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2958936-36ae-44cc-ad7a-c4c6c529a32c admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:09.840936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8764d64-d10a-494a-b066-f95b60b013c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:09.841681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8764d64-d10a-494a-b066-f95b60b013c8 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:09.841784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8764d64-d10a-494a-b066-f95b60b013c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:09.841968 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8764d64-d10a-494a-b066-f95b60b013c8 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:09.842614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8764d64-d10a-494a-b066-f95b60b013c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:09.844547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8764d64-d10a-494a-b066-f95b60b013c8 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:09.949212 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 940/1905] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:09 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:57:09.950264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8764d64-d10a-494a-b066-f95b60b013c8 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:09.961320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a283b3df-26fc-4a52-8281-214201f2757d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6887f447d084fb5a7f37378474142cc'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:09.963436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8764d64-d10a-494a-b066-f95b60b013c8 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:09.969516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bb619-9c6d-4c42-b8e5-2b6f6ad94112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:09.973359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bb619-9c6d-4c42-b8e5-2b6f6ad94112 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:09.973359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bb619-9c6d-4c42-b8e5-2b6f6ad94112 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:09.973359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bb619-9c6d-4c42-b8e5-2b6f6ad94112 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:09.973359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795bb619-9c6d-4c42-b8e5-2b6f6ad94112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:09.974955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795bb619-9c6d-4c42-b8e5-2b6f6ad94112 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:09.990562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 941/1906] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:09.992011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bb619-9c6d-4c42-b8e5-2b6f6ad94112 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:10.002516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bb619-9c6d-4c42-b8e5-2b6f6ad94112 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:10.008002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badaebf8-fdfa-41c8-b29f-4bb421374fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:10.008714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badaebf8-fdfa-41c8-b29f-4bb421374fbb admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:10.008999 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badaebf8-fdfa-41c8-b29f-4bb421374fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:10.009328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badaebf8-fdfa-41c8-b29f-4bb421374fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:10.010063 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badaebf8-fdfa-41c8-b29f-4bb421374fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:10.011756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badaebf8-fdfa-41c8-b29f-4bb421374fbb admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:10.022739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 942/1907] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:10.026632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badaebf8-fdfa-41c8-b29f-4bb421374fbb admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:10.040845 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a283b3df-26fc-4a52-8281-214201f2757d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6887f447d084fb5a7f37378474142cc'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:10.045039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a283b3df-26fc-4a52-8281-214201f2757d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6887f447d084fb5a7f37378474142cc'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:10.045039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a283b3df-26fc-4a52-8281-214201f2757d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6887f447d084fb5a7f37378474142cc'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:10.045471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badaebf8-fdfa-41c8-b29f-4bb421374fbb admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:10.055239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a50f3-0747-4217-8eba-3675120a4ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:10.055907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a50f3-0747-4217-8eba-3675120a4ef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:10.056236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a50f3-0747-4217-8eba-3675120a4ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:10.058648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a50f3-0747-4217-8eba-3675120a4ef9 admin admin] PATH_INFO: `/v3/projects/be78f395508b4dccaeaa34398d9c7218/users/c47943c3ea854811aed0261f6da81464/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:10.066727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a283b3df-26fc-4a52-8281-214201f2757d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6887f447d084fb5a7f37378474142cc'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:10.083975 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a50f3-0747-4217-8eba-3675120a4ef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be78f395508b4dccaeaa34398d9c7218, user_id=c47943c3ea854811aed0261f6da81464, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:10.086833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a50f3-0747-4217-8eba-3675120a4ef9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:10.088942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 966/1908] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:09 2022] DELETE /identity/v3/users/e6887f447d084fb5a7f37378474142cc => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:10.093852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c8cd6-3244-4ac0-9da2-bb8d8ffdfb8f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:10.094131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c8cd6-3244-4ac0-9da2-bb8d8ffdfb8f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:10.094402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c8cd6-3244-4ac0-9da2-bb8d8ffdfb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:10.114674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-758c8cd6-3244-4ac0-9da2-bb8d8ffdfb8f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:10.115873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-758c8cd6-3244-4ac0-9da2-bb8d8ffdfb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:10.183473 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 967/1909] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:10.198598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 943/1910] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:10 2022] PUT /identity/v3/projects/be78f395508b4dccaeaa34398d9c7218/users/c47943c3ea854811aed0261f6da81464/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:10.202928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc985c-ef12-4646-9dbe-4e4243bcdc15 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:10.203411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc985c-ef12-4646-9dbe-4e4243bcdc15 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:10.203887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc985c-ef12-4646-9dbe-4e4243bcdc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:10.216797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a50f3-0747-4217-8eba-3675120a4ef9 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:10.219045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-dd6a50f3-0747-4217-8eba-3675120a4ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:10.276157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0ecc985c-ef12-4646-9dbe-4e4243bcdc15 None None] MFA Rules not processed for user `c47943c3ea854811aed0261f6da81464`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:10.277798 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0ecc985c-ef12-4646-9dbe-4e4243bcdc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:10.344075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a50f3-0747-4217-8eba-3675120a4ef9 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:10.347815 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd1028-bc21-465a-adad-ea6137a5290a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:10.348503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1028-bc21-465a-adad-ea6137a5290a service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:10.348783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1028-bc21-465a-adad-ea6137a5290a service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:10.349054 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1028-bc21-465a-adad-ea6137a5290a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:10.376533 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 968/1911] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:10.382691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc985c-ef12-4646-9dbe-4e4243bcdc15 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:10.391236 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0ecc985c-ef12-4646-9dbe-4e4243bcdc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:10.435215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd1028-bc21-465a-adad-ea6137a5290a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:10.439492 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd1028-bc21-465a-adad-ea6137a5290a service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:10.482134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 944/1912] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:57:10.507188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc985c-ef12-4646-9dbe-4e4243bcdc15 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:10.516253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453c6f6-1aba-47c0-bf1d-b2f172dba365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:10.520677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453c6f6-1aba-47c0-bf1d-b2f172dba365 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:10.520677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453c6f6-1aba-47c0-bf1d-b2f172dba365 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:10.520677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453c6f6-1aba-47c0-bf1d-b2f172dba365 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:10.520677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453c6f6-1aba-47c0-bf1d-b2f172dba365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:10.520677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453c6f6-1aba-47c0-bf1d-b2f172dba365 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:10.664001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 969/1913] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:10 2022] POST /identity/v3/projects => generated 377 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:57:10.664573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd1028-bc21-465a-adad-ea6137a5290a service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:10.693104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd1028-bc21-465a-adad-ea6137a5290a service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:10.813467 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661efc9f-0ee1-4b43-b5bc-c2c874c7de80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:10.814112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661efc9f-0ee1-4b43-b5bc-c2c874c7de80 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:10.814385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661efc9f-0ee1-4b43-b5bc-c2c874c7de80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:10.818926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661efc9f-0ee1-4b43-b5bc-c2c874c7de80 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:10.818926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661efc9f-0ee1-4b43-b5bc-c2c874c7de80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:10.818926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661efc9f-0ee1-4b43-b5bc-c2c874c7de80 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:10.843924 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:11.055005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 945/1914] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:10 2022] POST /identity/v3/users => generated 295 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:11.057906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453c6f6-1aba-47c0-bf1d-b2f172dba365 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:11.076371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453c6f6-1aba-47c0-bf1d-b2f172dba365 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:11.088958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0fb38-6941-4200-9ce6-83fc4e64b34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:11.088958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0fb38-6941-4200-9ce6-83fc4e64b34d admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:11.088958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0fb38-6941-4200-9ce6-83fc4e64b34d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:11.088958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0fb38-6941-4200-9ce6-83fc4e64b34d admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:11.089780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0fb38-6941-4200-9ce6-83fc4e64b34d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:11.089780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0fb38-6941-4200-9ce6-83fc4e64b34d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:11.105689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 970/1915] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:11.105890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661efc9f-0ee1-4b43-b5bc-c2c874c7de80 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:11.117609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661efc9f-0ee1-4b43-b5bc-c2c874c7de80 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:11.138047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec9354-a98a-45e2-8eb9-3e28819b0cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:11.139218 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec9354-a98a-45e2-8eb9-3e28819b0cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:11.139565 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec9354-a98a-45e2-8eb9-3e28819b0cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:11.139947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec9354-a98a-45e2-8eb9-3e28819b0cb6 admin admin] PATH_INFO: `/v3/projects/253f525611884991be947ca6c921f44a/users/c074ff5ac60d45b2bea5ac36060b1d17/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:11.163545 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec9354-a98a-45e2-8eb9-3e28819b0cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=253f525611884991be947ca6c921f44a, user_id=c074ff5ac60d45b2bea5ac36060b1d17, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:11.167554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec9354-a98a-45e2-8eb9-3e28819b0cb6 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:11.287308 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 946/1916] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:11 2022] PUT /identity/v3/projects/253f525611884991be947ca6c921f44a/users/c074ff5ac60d45b2bea5ac36060b1d17/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:11.289072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0fb38-6941-4200-9ce6-83fc4e64b34d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:11.363595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0fb38-6941-4200-9ce6-83fc4e64b34d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:11.410850 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec9354-a98a-45e2-8eb9-3e28819b0cb6 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:11.412700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-56ec9354-a98a-45e2-8eb9-3e28819b0cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:11.450218 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df952a0a-0b13-4b77-b79f-08424801b813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:11.450446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df952a0a-0b13-4b77-b79f-08424801b813 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:11.450728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df952a0a-0b13-4b77-b79f-08424801b813 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:11.451002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df952a0a-0b13-4b77-b79f-08424801b813 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:11.451752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df952a0a-0b13-4b77-b79f-08424801b813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:11.453692 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df952a0a-0b13-4b77-b79f-08424801b813 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:11.478530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 971/1917] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:11.482669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df952a0a-0b13-4b77-b79f-08424801b813 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:11.493339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df952a0a-0b13-4b77-b79f-08424801b813 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:11.498859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f62ed-44a7-4f69-8b9d-da4da999be8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:11.499548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f62ed-44a7-4f69-8b9d-da4da999be8e admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:11.499806 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f62ed-44a7-4f69-8b9d-da4da999be8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:11.500453 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f62ed-44a7-4f69-8b9d-da4da999be8e admin admin] PATH_INFO: `/v3/projects/253f525611884991be947ca6c921f44a/users/c074ff5ac60d45b2bea5ac36060b1d17/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:11.502795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f62ed-44a7-4f69-8b9d-da4da999be8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=253f525611884991be947ca6c921f44a, user_id=c074ff5ac60d45b2bea5ac36060b1d17, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:11.505507 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f62ed-44a7-4f69-8b9d-da4da999be8e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:11.522841 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 972/1918] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:11 2022] PUT /identity/v3/projects/253f525611884991be947ca6c921f44a/users/c074ff5ac60d45b2bea5ac36060b1d17/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:11.534561 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8813b-a9c3-4c37-b670-690f6e1464da None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:11.535446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8813b-a9c3-4c37-b670-690f6e1464da None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:11.535748 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8813b-a9c3-4c37-b670-690f6e1464da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:11.540508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec9354-a98a-45e2-8eb9-3e28819b0cb6 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:11.574514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-57a8813b-a9c3-4c37-b670-690f6e1464da None None] MFA Rules not processed for user `c074ff5ac60d45b2bea5ac36060b1d17`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:11.578530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-57a8813b-a9c3-4c37-b670-690f6e1464da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:11.670031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:11.671186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:11.672083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:11.672377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] PATH_INFO: `/v3/projects/8e42a6903eea49faaf9ee72f0de0505c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:11.673858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e42a6903eea49faaf9ee72f0de0505c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:11.675601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 973/1919] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:11.680386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:11.682308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8813b-a9c3-4c37-b670-690f6e1464da None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:11.684010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-57a8813b-a9c3-4c37-b670-690f6e1464da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:11.742016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8813b-a9c3-4c37-b670-690f6e1464da None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:11.745768 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a17c2-5493-44e8-a491-bf053513f904 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:11.746371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a17c2-5493-44e8-a491-bf053513f904 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:11.746682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a17c2-5493-44e8-a491-bf053513f904 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:11.746957 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a17c2-5493-44e8-a491-bf053513f904 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:11.756675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a17c2-5493-44e8-a491-bf053513f904 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:11.758133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a17c2-5493-44e8-a491-bf053513f904 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:11.768613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 974/1920] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:57:11.801950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e42a6903eea49faaf9ee72f0de0505c', 'request_id': 'req-f058f312-ba55-46d9-936b-92d0db027485'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:11.808052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e42a6903eea49faaf9ee72f0de0505c', 'request_id': 'req-f058f312-ba55-46d9-936b-92d0db027485'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:11.926307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a17c2-5493-44e8-a491-bf053513f904 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:11.926547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e42a6903eea49faaf9ee72f0de0505c', 'request_id': 'req-f058f312-ba55-46d9-936b-92d0db027485'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:11.928205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-623a17c2-5493-44e8-a491-bf053513f904 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:11.938845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] The token cache is being invalidate because project 8e42a6903eea49faaf9ee72f0de0505c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:11.939138 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:11.939837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:11.942337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 947/1921] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:11 2022] DELETE /identity/v3/projects/8e42a6903eea49faaf9ee72f0de0505c => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:11.946470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a17c2-5493-44e8-a491-bf053513f904 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:11.946629 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:11.947991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:12.039271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80590f09-0a6d-4eb5-839a-ceb7177f9015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:12.039893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80590f09-0a6d-4eb5-839a-ceb7177f9015 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:12.040198 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80590f09-0a6d-4eb5-839a-ceb7177f9015 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:12.040473 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80590f09-0a6d-4eb5-839a-ceb7177f9015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:12.042724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-80590f09-0a6d-4eb5-839a-ceb7177f9015 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:12.105749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058f312-ba55-46d9-936b-92d0db027485 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:12.117873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c06007-b0a1-4803-9775-f364e6ec26c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:12.118770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c06007-b0a1-4803-9775-f364e6ec26c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:12.119123 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c06007-b0a1-4803-9775-f364e6ec26c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:12.119811 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c06007-b0a1-4803-9775-f364e6ec26c5 admin admin] PATH_INFO: `/v3/users/ca61bbfdb5804317b12080b40275cb18` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:12.123330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c06007-b0a1-4803-9775-f364e6ec26c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca61bbfdb5804317b12080b40275cb18)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:12.127574 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c06007-b0a1-4803-9775-f364e6ec26c5 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:12.182529 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80590f09-0a6d-4eb5-839a-ceb7177f9015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:12.184271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80590f09-0a6d-4eb5-839a-ceb7177f9015 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:12.195755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 975/1922] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:57:12.467361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c9c06007-b0a1-4803-9775-f364e6ec26c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca61bbfdb5804317b12080b40275cb18'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:12.477493 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:12.567386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c9c06007-b0a1-4803-9775-f364e6ec26c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca61bbfdb5804317b12080b40275cb18'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:12.567946 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c9c06007-b0a1-4803-9775-f364e6ec26c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca61bbfdb5804317b12080b40275cb18'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:12.568538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c9c06007-b0a1-4803-9775-f364e6ec26c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca61bbfdb5804317b12080b40275cb18'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:12.576100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c9c06007-b0a1-4803-9775-f364e6ec26c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca61bbfdb5804317b12080b40275cb18'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:12.604102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 948/1923] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:11 2022] DELETE /identity/v3/users/ca61bbfdb5804317b12080b40275cb18 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:13.100951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80590f09-0a6d-4eb5-839a-ceb7177f9015 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:13.122620 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80590f09-0a6d-4eb5-839a-ceb7177f9015 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:13.205185 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e03f6-4269-47a0-aeca-f70d63210f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:13.205786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e03f6-4269-47a0-aeca-f70d63210f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:13.205920 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e03f6-4269-47a0-aeca-f70d63210f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:13.206078 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e03f6-4269-47a0-aeca-f70d63210f72 admin admin] PATH_INFO: `/v3/projects/4a272e111c1c4c96beaa4620313d0f39` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:13.208328 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e03f6-4269-47a0-aeca-f70d63210f72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a272e111c1c4c96beaa4620313d0f39)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:13.210604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e03f6-4269-47a0-aeca-f70d63210f72 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:13.370145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-de7e03f6-4269-47a0-aeca-f70d63210f72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a272e111c1c4c96beaa4620313d0f39', 'request_id': 'req-de7e03f6-4269-47a0-aeca-f70d63210f72'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:13.387119 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-de7e03f6-4269-47a0-aeca-f70d63210f72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a272e111c1c4c96beaa4620313d0f39', 'request_id': 'req-de7e03f6-4269-47a0-aeca-f70d63210f72'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:13.533656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-de7e03f6-4269-47a0-aeca-f70d63210f72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a272e111c1c4c96beaa4620313d0f39', 'request_id': 'req-de7e03f6-4269-47a0-aeca-f70d63210f72'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:13.545841 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-de7e03f6-4269-47a0-aeca-f70d63210f72 admin admin] The token cache is being invalidate because project 4a272e111c1c4c96beaa4620313d0f39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:13.546136 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-de7e03f6-4269-47a0-aeca-f70d63210f72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:13.547067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-de7e03f6-4269-47a0-aeca-f70d63210f72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:13.548680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 976/1924] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:13 2022] DELETE /identity/v3/projects/4a272e111c1c4c96beaa4620313d0f39 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:13.554290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924494-cc72-444b-be8f-5d81f23f9fe9 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:13.554804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924494-cc72-444b-be8f-5d81f23f9fe9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:13.555351 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924494-cc72-444b-be8f-5d81f23f9fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:13.575446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-32924494-cc72-444b-be8f-5d81f23f9fe9 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:13.576766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-32924494-cc72-444b-be8f-5d81f23f9fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:13.642416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 949/1925] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:13.650636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595750da-5f2a-4639-8861-1b9b26cec1c7 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:13.651724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595750da-5f2a-4639-8861-1b9b26cec1c7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:13.652388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595750da-5f2a-4639-8861-1b9b26cec1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:13.679338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-595750da-5f2a-4639-8861-1b9b26cec1c7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:13.680919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-595750da-5f2a-4639-8861-1b9b26cec1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:13.689299 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 977/1926] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:13.694279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32924494-cc72-444b-be8f-5d81f23f9fe9 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:13.710893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32924494-cc72-444b-be8f-5d81f23f9fe9 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:13.710893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5e59c-3022-4a16-b223-38aee5d0714b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:13.711119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5e59c-3022-4a16-b223-38aee5d0714b admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:13.711572 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5e59c-3022-4a16-b223-38aee5d0714b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:13.711987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5e59c-3022-4a16-b223-38aee5d0714b admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:13.712771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5e59c-3022-4a16-b223-38aee5d0714b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:13.713214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5e59c-3022-4a16-b223-38aee5d0714b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:13.715025 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5e59c-3022-4a16-b223-38aee5d0714b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:13.726747 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 950/1927] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:13.731054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595750da-5f2a-4639-8861-1b9b26cec1c7 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:13.741988 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595750da-5f2a-4639-8861-1b9b26cec1c7 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:13.745604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614bc12-1518-4ef4-9627-99d7231ac31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:13.746195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614bc12-1518-4ef4-9627-99d7231ac31a admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:13.746473 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614bc12-1518-4ef4-9627-99d7231ac31a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:13.746777 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614bc12-1518-4ef4-9627-99d7231ac31a admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:13.747404 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614bc12-1518-4ef4-9627-99d7231ac31a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:13.749339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614bc12-1518-4ef4-9627-99d7231ac31a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:13.858016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 978/1928] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:13 2022] POST /identity/v3/projects => generated 393 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:13.859284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5e59c-3022-4a16-b223-38aee5d0714b admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:13.872303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5e59c-3022-4a16-b223-38aee5d0714b admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:13.952147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197ef67-a3f7-4c7f-a8c8-fb6815a4a1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:13.953043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197ef67-a3f7-4c7f-a8c8-fb6815a4a1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:13.953325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197ef67-a3f7-4c7f-a8c8-fb6815a4a1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:13.953657 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197ef67-a3f7-4c7f-a8c8-fb6815a4a1f2 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:13.954298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197ef67-a3f7-4c7f-a8c8-fb6815a4a1f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:13.956307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197ef67-a3f7-4c7f-a8c8-fb6815a4a1f2 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:14.058736 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 951/1929] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:13 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:14.062611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614bc12-1518-4ef4-9627-99d7231ac31a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:14.077755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614bc12-1518-4ef4-9627-99d7231ac31a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:14.100911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6131ff-8d30-4a0a-b69d-3cc87551ec09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:14.103296 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6131ff-8d30-4a0a-b69d-3cc87551ec09 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:14.104060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6131ff-8d30-4a0a-b69d-3cc87551ec09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:14.104560 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6131ff-8d30-4a0a-b69d-3cc87551ec09 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:14.105449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6131ff-8d30-4a0a-b69d-3cc87551ec09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:14.107553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6131ff-8d30-4a0a-b69d-3cc87551ec09 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:14.121486 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 979/1930] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:14.127306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197ef67-a3f7-4c7f-a8c8-fb6815a4a1f2 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:14.141671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197ef67-a3f7-4c7f-a8c8-fb6815a4a1f2 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:14.152659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdadc5-f4f0-43a2-929d-64e1c05f0adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:14.153226 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdadc5-f4f0-43a2-929d-64e1c05f0adc admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:14.153509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdadc5-f4f0-43a2-929d-64e1c05f0adc admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:14.153784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdadc5-f4f0-43a2-929d-64e1c05f0adc admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:14.154777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdadc5-f4f0-43a2-929d-64e1c05f0adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:14.156435 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdadc5-f4f0-43a2-929d-64e1c05f0adc admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:14.173146 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 952/1931] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:14.177974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6131ff-8d30-4a0a-b69d-3cc87551ec09 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:14.189194 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6131ff-8d30-4a0a-b69d-3cc87551ec09 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:14.194821 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf49d15-5afd-467b-8ad7-21d38551c579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:14.195500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf49d15-5afd-467b-8ad7-21d38551c579 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:14.195784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf49d15-5afd-467b-8ad7-21d38551c579 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:14.196053 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf49d15-5afd-467b-8ad7-21d38551c579 admin admin] PATH_INFO: `/v3/projects/42d1aa5aaceb43aa868cdd6f560d75f2/users/212bf053b439448cac50c96fc1afae98/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:14.212910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf49d15-5afd-467b-8ad7-21d38551c579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d1aa5aaceb43aa868cdd6f560d75f2, user_id=212bf053b439448cac50c96fc1afae98, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:14.218689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf49d15-5afd-467b-8ad7-21d38551c579 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:14.317436 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 980/1932] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:14 2022] PUT /identity/v3/projects/42d1aa5aaceb43aa868cdd6f560d75f2/users/212bf053b439448cac50c96fc1afae98/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:14.328522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81327ac3-e24c-4851-be12-ac61d348e722 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:14.328812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81327ac3-e24c-4851-be12-ac61d348e722 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:14.329090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81327ac3-e24c-4851-be12-ac61d348e722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:14.360986 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-81327ac3-e24c-4851-be12-ac61d348e722 None None] MFA Rules not processed for user `212bf053b439448cac50c96fc1afae98`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:14.362195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-81327ac3-e24c-4851-be12-ac61d348e722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:14.367206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:14.432446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 953/1933] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:14.437062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf49d15-5afd-467b-8ad7-21d38551c579 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:14.507157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf49d15-5afd-467b-8ad7-21d38551c579 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:14.513651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf1b04-0ec5-4bbf-a9fa-f42b3f17007a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:14.513651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf1b04-0ec5-4bbf-a9fa-f42b3f17007a admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:14.513651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf1b04-0ec5-4bbf-a9fa-f42b3f17007a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:14.513651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf1b04-0ec5-4bbf-a9fa-f42b3f17007a admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:14.514131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf1b04-0ec5-4bbf-a9fa-f42b3f17007a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:14.516098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf1b04-0ec5-4bbf-a9fa-f42b3f17007a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:14.655006 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 981/1934] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:14 2022] POST /identity/v3/projects => generated 393 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:14.656158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81327ac3-e24c-4851-be12-ac61d348e722 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:14.674955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81327ac3-e24c-4851-be12-ac61d348e722 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:14.757439 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a61291-2d72-47c4-9756-2ec519057b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:14.758402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a61291-2d72-47c4-9756-2ec519057b23 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:14.758748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a61291-2d72-47c4-9756-2ec519057b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:14.759013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a61291-2d72-47c4-9756-2ec519057b23 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:14.759718 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a61291-2d72-47c4-9756-2ec519057b23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:14.761713 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a61291-2d72-47c4-9756-2ec519057b23 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:14.887806 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf1b04-0ec5-4bbf-a9fa-f42b3f17007a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:14.890382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0adf1b04-0ec5-4bbf-a9fa-f42b3f17007a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:14.971331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf1b04-0ec5-4bbf-a9fa-f42b3f17007a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:14.975629 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a1a2b-b38f-41e5-bd5a-b1d80acc3151 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:14.976226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a1a2b-b38f-41e5-bd5a-b1d80acc3151 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:14.976475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a1a2b-b38f-41e5-bd5a-b1d80acc3151 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:14.976743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a1a2b-b38f-41e5-bd5a-b1d80acc3151 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:14.978435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-165a1a2b-b38f-41e5-bd5a-b1d80acc3151 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:15.007194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 954/1935] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:14 2022] POST /identity/v3/users => generated 303 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:57:15.011071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a61291-2d72-47c4-9756-2ec519057b23 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:15.021805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a61291-2d72-47c4-9756-2ec519057b23 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:15.027430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accef4c0-f6c5-4ec1-a0cd-45a22950ce1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:15.028155 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accef4c0-f6c5-4ec1-a0cd-45a22950ce1d admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:15.028492 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accef4c0-f6c5-4ec1-a0cd-45a22950ce1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:15.028844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accef4c0-f6c5-4ec1-a0cd-45a22950ce1d admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:15.044012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accef4c0-f6c5-4ec1-a0cd-45a22950ce1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:15.045144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accef4c0-f6c5-4ec1-a0cd-45a22950ce1d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:15.053035 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a1a2b-b38f-41e5-bd5a-b1d80acc3151 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:15.054873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a1a2b-b38f-41e5-bd5a-b1d80acc3151 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:15.065057 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 982/1936] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:57:15.073464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 955/1937] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:15.078715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a1a2b-b38f-41e5-bd5a-b1d80acc3151 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:15.096787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a1a2b-b38f-41e5-bd5a-b1d80acc3151 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:15.104072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74db7b8-e4a5-46fa-a006-2b1e760eb10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:15.104853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74db7b8-e4a5-46fa-a006-2b1e760eb10a admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:15.105410 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74db7b8-e4a5-46fa-a006-2b1e760eb10a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:15.105977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74db7b8-e4a5-46fa-a006-2b1e760eb10a admin admin] PATH_INFO: `/v3/projects/6e827c84813c4d0a8d3eff37ec8ddf81/users/f55189270fbf4eb58cf2124c2ecc8c61/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:15.121912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:15.132709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74db7b8-e4a5-46fa-a006-2b1e760eb10a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e827c84813c4d0a8d3eff37ec8ddf81, user_id=f55189270fbf4eb58cf2124c2ecc8c61, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:15.135782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74db7b8-e4a5-46fa-a006-2b1e760eb10a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:15.260912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 983/1938] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:15 2022] PUT /identity/v3/projects/6e827c84813c4d0a8d3eff37ec8ddf81/users/f55189270fbf4eb58cf2124c2ecc8c61/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:15.263310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accef4c0-f6c5-4ec1-a0cd-45a22950ce1d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:15.275111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accef4c0-f6c5-4ec1-a0cd-45a22950ce1d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:15.339259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14db4f7-83b8-4678-b993-5bb1a4b067fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:15.339859 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14db4f7-83b8-4678-b993-5bb1a4b067fb admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:15.340141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14db4f7-83b8-4678-b993-5bb1a4b067fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:15.340416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14db4f7-83b8-4678-b993-5bb1a4b067fb admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:15.341049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14db4f7-83b8-4678-b993-5bb1a4b067fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:15.342692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14db4f7-83b8-4678-b993-5bb1a4b067fb admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:15.353190 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 956/1939] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:15.357560 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74db7b8-e4a5-46fa-a006-2b1e760eb10a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:15.368118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74db7b8-e4a5-46fa-a006-2b1e760eb10a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:15.372822 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dd202-58bf-4f9b-a1e9-48f192ce8418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:15.373469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd202-58bf-4f9b-a1e9-48f192ce8418 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:15.373798 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd202-58bf-4f9b-a1e9-48f192ce8418 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:15.374005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd202-58bf-4f9b-a1e9-48f192ce8418 admin admin] PATH_INFO: `/v3/projects/6e827c84813c4d0a8d3eff37ec8ddf81/users/f55189270fbf4eb58cf2124c2ecc8c61/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:15.376230 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8dd202-58bf-4f9b-a1e9-48f192ce8418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e827c84813c4d0a8d3eff37ec8ddf81, user_id=f55189270fbf4eb58cf2124c2ecc8c61, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:15.395693 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8dd202-58bf-4f9b-a1e9-48f192ce8418 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:15.395693 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 984/1940] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:15 2022] PUT /identity/v3/projects/6e827c84813c4d0a8d3eff37ec8ddf81/users/f55189270fbf4eb58cf2124c2ecc8c61/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:15.415365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeee51-5615-4b73-ab8a-56bd2c7bcc29 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:15.415365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeee51-5615-4b73-ab8a-56bd2c7bcc29 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:15.415365 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeee51-5615-4b73-ab8a-56bd2c7bcc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:15.440418 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-b1eeee51-5615-4b73-ab8a-56bd2c7bcc29 None None] MFA Rules not processed for user `f55189270fbf4eb58cf2124c2ecc8c61`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:15.442165 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b1eeee51-5615-4b73-ab8a-56bd2c7bcc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:15.511108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 957/1941] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:15 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:15.645275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dd202-58bf-4f9b-a1e9-48f192ce8418 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:15.647059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ed8dd202-58bf-4f9b-a1e9-48f192ce8418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:15.705505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dd202-58bf-4f9b-a1e9-48f192ce8418 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:15.709494 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eccce4-e182-4012-a4f3-7623676c1d62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:15.712367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eccce4-e182-4012-a4f3-7623676c1d62 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:15.712367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eccce4-e182-4012-a4f3-7623676c1d62 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:15.712367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eccce4-e182-4012-a4f3-7623676c1d62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:15.719737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eccce4-e182-4012-a4f3-7623676c1d62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:15.721142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eccce4-e182-4012-a4f3-7623676c1d62 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:15.731349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 985/1942] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:15 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:15.902270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eeee51-5615-4b73-ab8a-56bd2c7bcc29 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:15.904845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b1eeee51-5615-4b73-ab8a-56bd2c7bcc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:15.904967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eccce4-e182-4012-a4f3-7623676c1d62 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:15.914570 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eccce4-e182-4012-a4f3-7623676c1d62 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:15.919319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707adbf6-fed0-4bad-9855-d3f67530eb25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:15.919648 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707adbf6-fed0-4bad-9855-d3f67530eb25 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:15.919922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707adbf6-fed0-4bad-9855-d3f67530eb25 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:15.920246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707adbf6-fed0-4bad-9855-d3f67530eb25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:16.017493 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707adbf6-fed0-4bad-9855-d3f67530eb25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:16.019652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707adbf6-fed0-4bad-9855-d3f67530eb25 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:16.027222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eeee51-5615-4b73-ab8a-56bd2c7bcc29 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:16.029572 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 986/1943] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:57:16.032593 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:16.033334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:16.033624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:16.033899 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] PATH_INFO: `/v3/users/4cc9d272d6714ad7aac0e208a8ed1175` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:16.035915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc9d272d6714ad7aac0e208a8ed1175)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:16.039301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:16.330249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc9d272d6714ad7aac0e208a8ed1175'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:16.418998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc9d272d6714ad7aac0e208a8ed1175'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:16.420383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc9d272d6714ad7aac0e208a8ed1175'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:16.422882 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc9d272d6714ad7aac0e208a8ed1175'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:16.439175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc9d272d6714ad7aac0e208a8ed1175'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:16.465346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 958/1944] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:15 2022] DELETE /identity/v3/users/4cc9d272d6714ad7aac0e208a8ed1175 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:16.475391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90248f8-0caa-4382-a9ac-d630d4c46c63 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:16.475925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90248f8-0caa-4382-a9ac-d630d4c46c63 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:16.476106 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90248f8-0caa-4382-a9ac-d630d4c46c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:16.497056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-e90248f8-0caa-4382-a9ac-d630d4c46c63 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:16.498710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e90248f8-0caa-4382-a9ac-d630d4c46c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:16.547456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:16.573534 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 987/1945] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:16.583837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:16.585471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:16.651931 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d285c1-bb47-4024-b54d-ac9c35ceff87 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:16.660208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fae5b7-6be7-4c0e-b238-80ad6f618755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:16.663177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fae5b7-6be7-4c0e-b238-80ad6f618755 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:16.663177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fae5b7-6be7-4c0e-b238-80ad6f618755 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:16.663177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fae5b7-6be7-4c0e-b238-80ad6f618755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:16.675053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fae5b7-6be7-4c0e-b238-80ad6f618755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:16.677651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fae5b7-6be7-4c0e-b238-80ad6f618755 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:16.691848 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 959/1946] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:17.165041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90248f8-0caa-4382-a9ac-d630d4c46c63 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:17.166779 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e90248f8-0caa-4382-a9ac-d630d4c46c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:17.179622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:17.192115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90248f8-0caa-4382-a9ac-d630d4c46c63 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:17.198018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:17.198704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:17.198989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:17.199261 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] PATH_INFO: `/v3/projects/3a265ce8247f4c3aac708aff32f44df9` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:17.200690 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a265ce8247f4c3aac708aff32f44df9)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:17.204624 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:17.372054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a265ce8247f4c3aac708aff32f44df9', 'request_id': 'req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:17.379142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a265ce8247f4c3aac708aff32f44df9', 'request_id': 'req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:17.469349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a265ce8247f4c3aac708aff32f44df9', 'request_id': 'req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:17.484434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] The token cache is being invalidate because project 3a265ce8247f4c3aac708aff32f44df9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:17.485247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:17.485385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:17.487122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 988/1947] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:17 2022] DELETE /identity/v3/projects/3a265ce8247f4c3aac708aff32f44df9 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:17.491654 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fae5b7-6be7-4c0e-b238-80ad6f618755 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:17.505421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fae5b7-6be7-4c0e-b238-80ad6f618755 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:17.598865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:17.599717 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:17.600099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:17.600445 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] PATH_INFO: `/v3/users/6f8e2c9e7ad042ffb7af6f845d02ebd5` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:17.601641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f8e2c9e7ad042ffb7af6f845d02ebd5)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:17.603673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:17.829156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f8e2c9e7ad042ffb7af6f845d02ebd5'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:18.082523 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f8e2c9e7ad042ffb7af6f845d02ebd5'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:18.083404 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f8e2c9e7ad042ffb7af6f845d02ebd5'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:18.084180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f8e2c9e7ad042ffb7af6f845d02ebd5'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:18.093728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f8e2c9e7ad042ffb7af6f845d02ebd5'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:18.127919 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 960/1948] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:17 2022] DELETE /identity/v3/users/6f8e2c9e7ad042ffb7af6f845d02ebd5 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:18.267905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:18.867360 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:18.868983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:18.894958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:18.897757 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:18.907204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4718c-b777-45f9-928c-ac2f9ef6d45e admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:19.077781 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbb0f1-06b6-4c69-ac7d-b2293174f4b3 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:19.084155 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eeeb09-31a8-41dd-bc34-a234a714f87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:19.085082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eeeb09-31a8-41dd-bc34-a234a714f87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:19.085562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eeeb09-31a8-41dd-bc34-a234a714f87c admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:19.086071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eeeb09-31a8-41dd-bc34-a234a714f87c admin admin] PATH_INFO: `/v3/projects/ffbfbad479554ac0b415970f9bcf8d11` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:19.087633 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eeeb09-31a8-41dd-bc34-a234a714f87c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffbfbad479554ac0b415970f9bcf8d11)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:19.089242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71d73e-550d-4cd1-ba21-5f866f6258b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:19.090138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71d73e-550d-4cd1-ba21-5f866f6258b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:19.090291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eeeb09-31a8-41dd-bc34-a234a714f87c admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:19.090739 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71d73e-550d-4cd1-ba21-5f866f6258b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:19.091179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71d73e-550d-4cd1-ba21-5f866f6258b7 admin admin] PATH_INFO: `/v3/users/858f72c8c5ff49b6977bae6096bc881b` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:19.093584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71d73e-550d-4cd1-ba21-5f866f6258b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=858f72c8c5ff49b6977bae6096bc881b)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:19.095709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71d73e-550d-4cd1-ba21-5f866f6258b7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:19.260135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:19.291255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e2eeeb09-31a8-41dd-bc34-a234a714f87c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffbfbad479554ac0b415970f9bcf8d11', 'request_id': 'req-e2eeeb09-31a8-41dd-bc34-a234a714f87c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:19.307871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e2eeeb09-31a8-41dd-bc34-a234a714f87c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffbfbad479554ac0b415970f9bcf8d11', 'request_id': 'req-e2eeeb09-31a8-41dd-bc34-a234a714f87c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:19.373065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-3d71d73e-550d-4cd1-ba21-5f866f6258b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '858f72c8c5ff49b6977bae6096bc881b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:19.425775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e2eeeb09-31a8-41dd-bc34-a234a714f87c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffbfbad479554ac0b415970f9bcf8d11', 'request_id': 'req-e2eeeb09-31a8-41dd-bc34-a234a714f87c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:19.439177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e2eeeb09-31a8-41dd-bc34-a234a714f87c admin admin] The token cache is being invalidate because project ffbfbad479554ac0b415970f9bcf8d11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:19.439562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e2eeeb09-31a8-41dd-bc34-a234a714f87c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:19.440464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e2eeeb09-31a8-41dd-bc34-a234a714f87c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:19.443552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 961/1949] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:18 2022] DELETE /identity/v3/projects/ffbfbad479554ac0b415970f9bcf8d11 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:19.448718 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a6dd4-fc5a-4274-bbe1-4989a88f28ff None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:19.449085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a6dd4-fc5a-4274-bbe1-4989a88f28ff None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:19.449406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a6dd4-fc5a-4274-bbe1-4989a88f28ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:19.475286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-618a6dd4-fc5a-4274-bbe1-4989a88f28ff None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:19.476578 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-618a6dd4-fc5a-4274-bbe1-4989a88f28ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:19.512663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:19.554217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 962/1950] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:19.560538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-3d71d73e-550d-4cd1-ba21-5f866f6258b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '858f72c8c5ff49b6977bae6096bc881b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:19.564017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-3d71d73e-550d-4cd1-ba21-5f866f6258b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '858f72c8c5ff49b6977bae6096bc881b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:19.564017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-3d71d73e-550d-4cd1-ba21-5f866f6258b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '858f72c8c5ff49b6977bae6096bc881b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:19.564221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79e285-52b3-4bc2-8b14-53727a1e6931 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:19.564221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79e285-52b3-4bc2-8b14-53727a1e6931 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:19.564221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79e285-52b3-4bc2-8b14-53727a1e6931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:19.583341 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-3d71d73e-550d-4cd1-ba21-5f866f6258b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '858f72c8c5ff49b6977bae6096bc881b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:19.589381 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-1d79e285-52b3-4bc2-8b14-53727a1e6931 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:19.592720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-1d79e285-52b3-4bc2-8b14-53727a1e6931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:19.600065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 963/1951] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:19.604285 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79e285-52b3-4bc2-8b14-53727a1e6931 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:19.606400 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 989/1952] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:18 2022] DELETE /identity/v3/users/858f72c8c5ff49b6977bae6096bc881b => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:19.610771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab8103-854f-4fcb-829d-d30848150e76 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:19.611113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab8103-854f-4fcb-829d-d30848150e76 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:19.611443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab8103-854f-4fcb-829d-d30848150e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:19.613821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79e285-52b3-4bc2-8b14-53727a1e6931 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:19.629816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-4cab8103-854f-4fcb-829d-d30848150e76 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:19.631095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4cab8103-854f-4fcb-829d-d30848150e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:19.677087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e64960-9fca-4e9d-b12e-c1449057ac79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:19.677609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e64960-9fca-4e9d-b12e-c1449057ac79 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:19.677885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e64960-9fca-4e9d-b12e-c1449057ac79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:19.678160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e64960-9fca-4e9d-b12e-c1449057ac79 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:19.678974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e64960-9fca-4e9d-b12e-c1449057ac79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:19.679279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e64960-9fca-4e9d-b12e-c1449057ac79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:19.680866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e64960-9fca-4e9d-b12e-c1449057ac79 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:19.696675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 964/1953] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:57:19.701692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e64960-9fca-4e9d-b12e-c1449057ac79 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:19.727286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e64960-9fca-4e9d-b12e-c1449057ac79 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:19.731119 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 990/1954] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:19.736783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27af93-e615-4247-b3b3-18b696b53c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:19.737702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27af93-e615-4247-b3b3-18b696b53c82 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:19.738347 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27af93-e615-4247-b3b3-18b696b53c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:19.738805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27af93-e615-4247-b3b3-18b696b53c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:19.739653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27af93-e615-4247-b3b3-18b696b53c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:19.741903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27af93-e615-4247-b3b3-18b696b53c82 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:19.754199 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab8103-854f-4fcb-829d-d30848150e76 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:19.755578 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4cab8103-854f-4fcb-829d-d30848150e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:19.814351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab8103-854f-4fcb-829d-d30848150e76 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:19.818837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abaae4-1e37-420d-a370-2dd12d14e44b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:19.819445 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abaae4-1e37-420d-a370-2dd12d14e44b service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:19.819719 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abaae4-1e37-420d-a370-2dd12d14e44b service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:19.819988 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abaae4-1e37-420d-a370-2dd12d14e44b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:19.829578 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5abaae4-1e37-420d-a370-2dd12d14e44b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:19.831222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5abaae4-1e37-420d-a370-2dd12d14e44b service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:19.841388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 991/1955] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:19.858420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 965/1956] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:19 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:19.859818 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abaae4-1e37-420d-a370-2dd12d14e44b service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:19.870321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abaae4-1e37-420d-a370-2dd12d14e44b service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:19.943854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fc33f-62be-40ce-84be-7d5570d639f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:19.944508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fc33f-62be-40ce-84be-7d5570d639f4 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:19.944787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fc33f-62be-40ce-84be-7d5570d639f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:19.945060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fc33f-62be-40ce-84be-7d5570d639f4 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:19.945835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44fc33f-62be-40ce-84be-7d5570d639f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:19.947878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44fc33f-62be-40ce-84be-7d5570d639f4 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:20.047483 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 992/1957] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:19 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:20.076705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27af93-e615-4247-b3b3-18b696b53c82 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:20.103674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27af93-e615-4247-b3b3-18b696b53c82 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:20.103674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5f71a-4fa0-4f31-8acf-b0d27fd7e84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:20.103674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5f71a-4fa0-4f31-8acf-b0d27fd7e84d admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:20.103674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5f71a-4fa0-4f31-8acf-b0d27fd7e84d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:20.104388 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5f71a-4fa0-4f31-8acf-b0d27fd7e84d admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:20.104388 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5f71a-4fa0-4f31-8acf-b0d27fd7e84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:20.104388 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5f71a-4fa0-4f31-8acf-b0d27fd7e84d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:20.110860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 966/1958] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:20.114919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fc33f-62be-40ce-84be-7d5570d639f4 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:20.126641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fc33f-62be-40ce-84be-7d5570d639f4 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:20.132256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deac101-782d-4de2-a905-462ab132d3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:20.133075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deac101-782d-4de2-a905-462ab132d3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:20.133354 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deac101-782d-4de2-a905-462ab132d3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:20.133622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deac101-782d-4de2-a905-462ab132d3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:20.134264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deac101-782d-4de2-a905-462ab132d3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:20.135878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deac101-782d-4de2-a905-462ab132d3b7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:20.147005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 993/1959] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:20.151104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5f71a-4fa0-4f31-8acf-b0d27fd7e84d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:20.161814 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5f71a-4fa0-4f31-8acf-b0d27fd7e84d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:20.166645 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2418d-5d72-4b6f-b2b9-48b2bb14ad21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:20.167298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2418d-5d72-4b6f-b2b9-48b2bb14ad21 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:20.167577 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2418d-5d72-4b6f-b2b9-48b2bb14ad21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:20.167850 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2418d-5d72-4b6f-b2b9-48b2bb14ad21 admin admin] PATH_INFO: `/v3/projects/e7b789c30cf647c69e1a59bf3b99eda3/users/a683021de6904ca6b770d98a7051fb04/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:20.183173 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2418d-5d72-4b6f-b2b9-48b2bb14ad21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b789c30cf647c69e1a59bf3b99eda3, user_id=a683021de6904ca6b770d98a7051fb04, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:20.185763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2418d-5d72-4b6f-b2b9-48b2bb14ad21 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:20.260089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:20.355021 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deac101-782d-4de2-a905-462ab132d3b7 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:20.356007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6deac101-782d-4de2-a905-462ab132d3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:20.360604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 967/1960] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:20 2022] PUT /identity/v3/projects/e7b789c30cf647c69e1a59bf3b99eda3/users/a683021de6904ca6b770d98a7051fb04/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:20.362275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936d650-d22c-4c57-ac0c-d7f26c0605d7 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:20.363231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936d650-d22c-4c57-ac0c-d7f26c0605d7 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:20.363661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936d650-d22c-4c57-ac0c-d7f26c0605d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:20.394272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deac101-782d-4de2-a905-462ab132d3b7 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:20.410996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-e936d650-d22c-4c57-ac0c-d7f26c0605d7 None None] MFA Rules not processed for user `a683021de6904ca6b770d98a7051fb04`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:20.412832 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e936d650-d22c-4c57-ac0c-d7f26c0605d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:20.478138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:20.479416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:20.480050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:20.480647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] PATH_INFO: `/v3/projects/a71c46f52c924e4cb429335c950d43c4` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:20.482340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=a71c46f52c924e4cb429335c950d43c4)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:20.484601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:20.528933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 968/1961] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:20.530466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936d650-d22c-4c57-ac0c-d7f26c0605d7 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:20.543841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936d650-d22c-4c57-ac0c-d7f26c0605d7 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:20.547316 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38cceb-aa2b-4d62-847e-27962128af0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:20.547775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38cceb-aa2b-4d62-847e-27962128af0f admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:20.548055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38cceb-aa2b-4d62-847e-27962128af0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:20.548324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38cceb-aa2b-4d62-847e-27962128af0f admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:20.548944 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38cceb-aa2b-4d62-847e-27962128af0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:20.551486 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38cceb-aa2b-4d62-847e-27962128af0f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:20.626206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71c46f52c924e4cb429335c950d43c4', 'request_id': 'req-b6355ccc-c923-4492-9609-197bd2bf09ac'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:20.626364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:20.635007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71c46f52c924e4cb429335c950d43c4', 'request_id': 'req-b6355ccc-c923-4492-9609-197bd2bf09ac'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:20.648292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 969/1962] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:20 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:20.649397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38cceb-aa2b-4d62-847e-27962128af0f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:20.705161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71c46f52c924e4cb429335c950d43c4', 'request_id': 'req-b6355ccc-c923-4492-9609-197bd2bf09ac'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:20.717121 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] The token cache is being invalidate because project a71c46f52c924e4cb429335c950d43c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:20.717490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:20.718189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:20.719827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 994/1963] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:20 2022] DELETE /identity/v3/projects/a71c46f52c924e4cb429335c950d43c4 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:20.724037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:20.725582 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:20.726100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38cceb-aa2b-4d62-847e-27962128af0f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:20.731437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13941c-49cb-4c42-9d97-8e11d44e9729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:20.732306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13941c-49cb-4c42-9d97-8e11d44e9729 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:20.732587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13941c-49cb-4c42-9d97-8e11d44e9729 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:20.732913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13941c-49cb-4c42-9d97-8e11d44e9729 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:20.733560 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13941c-49cb-4c42-9d97-8e11d44e9729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:20.735566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13941c-49cb-4c42-9d97-8e11d44e9729 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:20.810233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6355ccc-c923-4492-9609-197bd2bf09ac admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:20.817481 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:20.818322 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:20.818754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:20.819135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] PATH_INFO: `/v3/users/c47943c3ea854811aed0261f6da81464` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:20.820477 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c47943c3ea854811aed0261f6da81464)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:20.825251 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:20.870985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 970/1964] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:20 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:20.872100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13941c-49cb-4c42-9d97-8e11d44e9729 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:20.873810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9c13941c-49cb-4c42-9d97-8e11d44e9729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:20.886051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13941c-49cb-4c42-9d97-8e11d44e9729 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:20.895662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2e8ef-3d60-49bc-8417-4b08d1c1d437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:20.895662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2e8ef-3d60-49bc-8417-4b08d1c1d437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:20.895662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2e8ef-3d60-49bc-8417-4b08d1c1d437 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:20.895662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2e8ef-3d60-49bc-8417-4b08d1c1d437 admin admin] PATH_INFO: `/v3/users/89e2d95af5ca4a659aac274b4f3568af` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:20.896013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2e8ef-3d60-49bc-8417-4b08d1c1d437 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89e2d95af5ca4a659aac274b4f3568af)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:20.896458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2e8ef-3d60-49bc-8417-4b08d1c1d437 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:21.198929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c47943c3ea854811aed0261f6da81464'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:21.239496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9dd2e8ef-3d60-49bc-8417-4b08d1c1d437 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89e2d95af5ca4a659aac274b4f3568af'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:21.276649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c47943c3ea854811aed0261f6da81464'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:21.278066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c47943c3ea854811aed0261f6da81464'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:21.278811 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c47943c3ea854811aed0261f6da81464'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:21.292631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c47943c3ea854811aed0261f6da81464'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:21.305870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 995/1965] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:20 2022] DELETE /identity/v3/users/c47943c3ea854811aed0261f6da81464 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:21.307174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:21.314082 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:21.367949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9dd2e8ef-3d60-49bc-8417-4b08d1c1d437 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89e2d95af5ca4a659aac274b4f3568af'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:21.368585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9dd2e8ef-3d60-49bc-8417-4b08d1c1d437 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89e2d95af5ca4a659aac274b4f3568af'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:21.369589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9dd2e8ef-3d60-49bc-8417-4b08d1c1d437 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89e2d95af5ca4a659aac274b4f3568af'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:21.378591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9dd2e8ef-3d60-49bc-8417-4b08d1c1d437 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89e2d95af5ca4a659aac274b4f3568af'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:21.403281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 971/1966] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:20 2022] DELETE /identity/v3/users/89e2d95af5ca4a659aac274b4f3568af => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:21.415579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01afa9-1cf1-4be7-baf9-fde6b02a8ab3 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:21.415742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee16333-e3b7-4895-a972-ab2755983f87 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:21.415742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee16333-e3b7-4895-a972-ab2755983f87 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:21.415742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee16333-e3b7-4895-a972-ab2755983f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:21.455344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-fee16333-e3b7-4895-a972-ab2755983f87 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:21.458270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-fee16333-e3b7-4895-a972-ab2755983f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:21.512931 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bd5ba-9048-4950-aa80-772324eaf5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:21.513479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bd5ba-9048-4950-aa80-772324eaf5fc admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:21.513757 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bd5ba-9048-4950-aa80-772324eaf5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:21.514431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bd5ba-9048-4950-aa80-772324eaf5fc admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:21.514717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7bd5ba-9048-4950-aa80-772324eaf5fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:21.516320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7bd5ba-9048-4950-aa80-772324eaf5fc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:21.531594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 996/1967] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:21.532780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bd5ba-9048-4950-aa80-772324eaf5fc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:21.534535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7b7bd5ba-9048-4950-aa80-772324eaf5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:21.539604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 972/1968] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:21.546367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee16333-e3b7-4895-a972-ab2755983f87 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:21.547838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-fee16333-e3b7-4895-a972-ab2755983f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:21.548419 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bd5ba-9048-4950-aa80-772324eaf5fc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:21.553723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b9c15-bd82-4ee6-af53-d2e3f13e8ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:21.554592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b9c15-bd82-4ee6-af53-d2e3f13e8ab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:21.554714 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b9c15-bd82-4ee6-af53-d2e3f13e8ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:21.554953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b9c15-bd82-4ee6-af53-d2e3f13e8ab0 admin admin] PATH_INFO: `/v3/projects/a0892bd82d6346f5b44e663d73692e2a/users/6b54198943de4072991f0e2feca855fa/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:21.572307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b9c15-bd82-4ee6-af53-d2e3f13e8ab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0892bd82d6346f5b44e663d73692e2a, user_id=6b54198943de4072991f0e2feca855fa, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:21.575369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b9c15-bd82-4ee6-af53-d2e3f13e8ab0 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:21.612616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee16333-e3b7-4895-a972-ab2755983f87 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:21.616391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467e6a8-1667-4c32-82b6-b61ec956b0d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:21.616997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467e6a8-1667-4c32-82b6-b61ec956b0d8 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:21.617571 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467e6a8-1667-4c32-82b6-b61ec956b0d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:21.618141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467e6a8-1667-4c32-82b6-b61ec956b0d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:21.628671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467e6a8-1667-4c32-82b6-b61ec956b0d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:21.630685 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467e6a8-1667-4c32-82b6-b61ec956b0d8 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:21.640466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 973/1969] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:21.659071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 997/1970] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:21 2022] PUT /identity/v3/projects/a0892bd82d6346f5b44e663d73692e2a/users/6b54198943de4072991f0e2feca855fa/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:21.663984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467e6a8-1667-4c32-82b6-b61ec956b0d8 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:21.674993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467e6a8-1667-4c32-82b6-b61ec956b0d8 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:21.762196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecc008-87d1-4790-b738-61a7f506511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:21.763632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecc008-87d1-4790-b738-61a7f506511b admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:21.764220 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecc008-87d1-4790-b738-61a7f506511b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:21.764220 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecc008-87d1-4790-b738-61a7f506511b admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:21.765122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecc008-87d1-4790-b738-61a7f506511b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:21.767497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecc008-87d1-4790-b738-61a7f506511b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:21.782360 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 974/1971] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:21.786640 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b9c15-bd82-4ee6-af53-d2e3f13e8ab0 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:21.808161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b9c15-bd82-4ee6-af53-d2e3f13e8ab0 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:21.818102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898f16-c6cd-4bbb-8341-88357f2e6dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:21.819431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898f16-c6cd-4bbb-8341-88357f2e6dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:21.819805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898f16-c6cd-4bbb-8341-88357f2e6dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:21.820181 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898f16-c6cd-4bbb-8341-88357f2e6dd7 admin admin] PATH_INFO: `/v3/projects/a0892bd82d6346f5b44e663d73692e2a/users/6b54198943de4072991f0e2feca855fa/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:21.824785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9898f16-c6cd-4bbb-8341-88357f2e6dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0892bd82d6346f5b44e663d73692e2a, user_id=6b54198943de4072991f0e2feca855fa, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:21.828009 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9898f16-c6cd-4bbb-8341-88357f2e6dd7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:21.841699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 998/1972] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:21 2022] PUT /identity/v3/projects/a0892bd82d6346f5b44e663d73692e2a/users/6b54198943de4072991f0e2feca855fa/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:21.854254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240a8e9-c435-43a8-95f8-687be23547c0 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:21.854680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240a8e9-c435-43a8-95f8-687be23547c0 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:21.855050 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240a8e9-c435-43a8-95f8-687be23547c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:21.891069 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-1240a8e9-c435-43a8-95f8-687be23547c0 None None] MFA Rules not processed for user `6b54198943de4072991f0e2feca855fa`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:21.893018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-1240a8e9-c435-43a8-95f8-687be23547c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:21.996684 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 975/1973] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:21.997725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898f16-c6cd-4bbb-8341-88357f2e6dd7 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:21.999562 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e9898f16-c6cd-4bbb-8341-88357f2e6dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:22.009610 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240a8e9-c435-43a8-95f8-687be23547c0 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:22.012205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-1240a8e9-c435-43a8-95f8-687be23547c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:22.043858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:22.090375 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898f16-c6cd-4bbb-8341-88357f2e6dd7 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:22.096985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1629a-fdb4-43f6-8a91-eec1a51636db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:22.097893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1629a-fdb4-43f6-8a91-eec1a51636db admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:22.098449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1629a-fdb4-43f6-8a91-eec1a51636db admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:22.099014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1629a-fdb4-43f6-8a91-eec1a51636db admin admin] PATH_INFO: `/v3/projects/be78f395508b4dccaeaa34398d9c7218` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:22.100454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1629a-fdb4-43f6-8a91-eec1a51636db admin admin] RBAC: Authorizing `identity:delete_project(project_id=be78f395508b4dccaeaa34398d9c7218)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:22.102707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1629a-fdb4-43f6-8a91-eec1a51636db admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:22.153687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240a8e9-c435-43a8-95f8-687be23547c0 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:22.158972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ba4a3-3c61-4113-bea1-cfdcbe72ffa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:22.160610 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ba4a3-3c61-4113-bea1-cfdcbe72ffa9 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:22.161143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ba4a3-3c61-4113-bea1-cfdcbe72ffa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:22.161521 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ba4a3-3c61-4113-bea1-cfdcbe72ffa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:22.179852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:22.181872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ba4a3-3c61-4113-bea1-cfdcbe72ffa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:22.184531 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ba4a3-3c61-4113-bea1-cfdcbe72ffa9 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:22.202304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 976/1974] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:22.242903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-cbe1629a-fdb4-43f6-8a91-eec1a51636db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be78f395508b4dccaeaa34398d9c7218', 'request_id': 'req-cbe1629a-fdb4-43f6-8a91-eec1a51636db'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.249223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-cbe1629a-fdb4-43f6-8a91-eec1a51636db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be78f395508b4dccaeaa34398d9c7218', 'request_id': 'req-cbe1629a-fdb4-43f6-8a91-eec1a51636db'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.353720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cdbca-3956-4c06-a05e-1587eea42e06 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:22.354310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cdbca-3956-4c06-a05e-1587eea42e06 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:22.354566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cdbca-3956-4c06-a05e-1587eea42e06 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:22.356232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 977/1975] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 12:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:57:22.356356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-cbe1629a-fdb4-43f6-8a91-eec1a51636db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be78f395508b4dccaeaa34398d9c7218', 'request_id': 'req-cbe1629a-fdb4-43f6-8a91-eec1a51636db'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.370822 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cdbca-3956-4c06-a05e-1587eea42e06 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:22.373806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-238cdbca-3956-4c06-a05e-1587eea42e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:22.376960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-cbe1629a-fdb4-43f6-8a91-eec1a51636db admin admin] The token cache is being invalidate because project be78f395508b4dccaeaa34398d9c7218 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:22.377412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-cbe1629a-fdb4-43f6-8a91-eec1a51636db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.379019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-cbe1629a-fdb4-43f6-8a91-eec1a51636db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.384738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 999/1976] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:21 2022] DELETE /identity/v3/projects/be78f395508b4dccaeaa34398d9c7218 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:22.386508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d65173-78f9-460e-8c1a-1d93d62ee3a0 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:22.386921 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d65173-78f9-460e-8c1a-1d93d62ee3a0 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:22.387301 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d65173-78f9-460e-8c1a-1d93d62ee3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:22.413784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-19d65173-78f9-460e-8c1a-1d93d62ee3a0 None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:22.415488 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-19d65173-78f9-460e-8c1a-1d93d62ee3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:22.493002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cdbca-3956-4c06-a05e-1587eea42e06 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:22.497845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:22.498708 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:22.499151 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:22.499587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1000/1977] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 12:57:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:22.499858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] PATH_INFO: `/v3/projects/49efc288d62f4ebb8173c59c3885f2fc` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:22.500448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d65173-78f9-460e-8c1a-1d93d62ee3a0 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:22.501286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49efc288d62f4ebb8173c59c3885f2fc)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:22.501772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-19d65173-78f9-460e-8c1a-1d93d62ee3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:22.504487 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:22.521510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d65173-78f9-460e-8c1a-1d93d62ee3a0 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:22.527575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:22.528219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:22.528509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:22.528823 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] PATH_INFO: `/v3/users/c074ff5ac60d45b2bea5ac36060b1d17` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:22.530067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c074ff5ac60d45b2bea5ac36060b1d17)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:22.538712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:22.734441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49efc288d62f4ebb8173c59c3885f2fc', 'request_id': 'req-57205c74-53ec-4632-adbe-758d37de4839'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.743656 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49efc288d62f4ebb8173c59c3885f2fc', 'request_id': 'req-57205c74-53ec-4632-adbe-758d37de4839'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.820048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49efc288d62f4ebb8173c59c3885f2fc', 'request_id': 'req-57205c74-53ec-4632-adbe-758d37de4839'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.855904 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c074ff5ac60d45b2bea5ac36060b1d17'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.860620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] The token cache is being invalidate because project 49efc288d62f4ebb8173c59c3885f2fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:22.861252 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.861896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.863960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 978/1978] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:22 2022] DELETE /identity/v3/projects/49efc288d62f4ebb8173c59c3885f2fc => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:22.864870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:22.869801 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:22.914560 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c074ff5ac60d45b2bea5ac36060b1d17'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.915444 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c074ff5ac60d45b2bea5ac36060b1d17'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.916057 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c074ff5ac60d45b2bea5ac36060b1d17'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.930132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c074ff5ac60d45b2bea5ac36060b1d17'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:22.947455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1001/1979] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:22 2022] DELETE /identity/v3/users/c074ff5ac60d45b2bea5ac36060b1d17 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:22.948634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:22.950278 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:22.954579 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:22.968636 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57205c74-53ec-4632-adbe-758d37de4839 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:23.069024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a204eee-be1d-4fd2-9cd6-8e9eccbc4085 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:23.089258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba290ad4-2705-4cf2-8d86-99c232d10bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:23.089903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07950075-6df9-4ce6-b0dc-6898628b7f94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:23.090550 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07950075-6df9-4ce6-b0dc-6898628b7f94 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:23.090838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07950075-6df9-4ce6-b0dc-6898628b7f94 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:23.091113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07950075-6df9-4ce6-b0dc-6898628b7f94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:23.091657 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba290ad4-2705-4cf2-8d86-99c232d10bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:23.092118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba290ad4-2705-4cf2-8d86-99c232d10bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:23.092560 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba290ad4-2705-4cf2-8d86-99c232d10bf4 admin admin] PATH_INFO: `/v3/users/212bf053b439448cac50c96fc1afae98` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:23.093104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-07950075-6df9-4ce6-b0dc-6898628b7f94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:23.094159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba290ad4-2705-4cf2-8d86-99c232d10bf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=212bf053b439448cac50c96fc1afae98)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:23.097271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba290ad4-2705-4cf2-8d86-99c232d10bf4 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:23.111082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07950075-6df9-4ce6-b0dc-6898628b7f94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:23.112936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07950075-6df9-4ce6-b0dc-6898628b7f94 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:23.130643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 979/1980] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:23.134904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07950075-6df9-4ce6-b0dc-6898628b7f94 service nova] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:23.134904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-07950075-6df9-4ce6-b0dc-6898628b7f94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:23.148836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07950075-6df9-4ce6-b0dc-6898628b7f94 service nova] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:23.194407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:23.195107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:23.195348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:23.198997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] PATH_INFO: `/v3/users/e05194319d16404e874a338c8f8ab919` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:23.200513 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e05194319d16404e874a338c8f8ab919)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:23.204767 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:23.248386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:23.322194 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ba290ad4-2705-4cf2-8d86-99c232d10bf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '212bf053b439448cac50c96fc1afae98'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:23.481615 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ba290ad4-2705-4cf2-8d86-99c232d10bf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '212bf053b439448cac50c96fc1afae98'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:23.492566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ba290ad4-2705-4cf2-8d86-99c232d10bf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '212bf053b439448cac50c96fc1afae98'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:23.492566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ba290ad4-2705-4cf2-8d86-99c232d10bf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '212bf053b439448cac50c96fc1afae98'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:23.505443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ba290ad4-2705-4cf2-8d86-99c232d10bf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '212bf053b439448cac50c96fc1afae98'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:23.512917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e05194319d16404e874a338c8f8ab919'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:23.526248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1002/1981] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:22 2022] DELETE /identity/v3/users/212bf053b439448cac50c96fc1afae98 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:23.532513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1901550-6a64-4f2d-a438-5255b94e0144 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:23.532784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1901550-6a64-4f2d-a438-5255b94e0144 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:23.533062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1901550-6a64-4f2d-a438-5255b94e0144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:23.568716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-f1901550-6a64-4f2d-a438-5255b94e0144 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:23.570092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f1901550-6a64-4f2d-a438-5255b94e0144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:23.590737 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e05194319d16404e874a338c8f8ab919'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:23.591419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e05194319d16404e874a338c8f8ab919'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:23.592234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e05194319d16404e874a338c8f8ab919'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:23.618189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e05194319d16404e874a338c8f8ab919'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:23.656054 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 980/1982] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:23 2022] DELETE /identity/v3/users/e05194319d16404e874a338c8f8ab919 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:23.656054 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:23.662823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:23.731758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1003/1983] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:23.733914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1901550-6a64-4f2d-a438-5255b94e0144 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:23.738755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f1901550-6a64-4f2d-a438-5255b94e0144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:23.761245 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dc302-9c7c-49b0-86b4-dc3ff9f452cf admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:23.766795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70302b70-dc8f-474f-ba7b-1f1c48d62079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:23.767514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70302b70-dc8f-474f-ba7b-1f1c48d62079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:23.773964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70302b70-dc8f-474f-ba7b-1f1c48d62079 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:23.773964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70302b70-dc8f-474f-ba7b-1f1c48d62079 admin admin] PATH_INFO: `/v3/users/a683021de6904ca6b770d98a7051fb04` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:23.773964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70302b70-dc8f-474f-ba7b-1f1c48d62079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a683021de6904ca6b770d98a7051fb04)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:23.773964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70302b70-dc8f-474f-ba7b-1f1c48d62079 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:23.795360 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1901550-6a64-4f2d-a438-5255b94e0144 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:23.801143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:23.802093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:23.802367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:23.802771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] PATH_INFO: `/v3/projects/253f525611884991be947ca6c921f44a` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:23.804042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=253f525611884991be947ca6c921f44a)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:23.806402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:23.995243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253f525611884991be947ca6c921f44a', 'request_id': 'req-4078b77e-6359-4a5c-bea3-845c845ae6e9'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.003048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253f525611884991be947ca6c921f44a', 'request_id': 'req-4078b77e-6359-4a5c-bea3-845c845ae6e9'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.009167 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-70302b70-dc8f-474f-ba7b-1f1c48d62079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a683021de6904ca6b770d98a7051fb04'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.099609 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253f525611884991be947ca6c921f44a', 'request_id': 'req-4078b77e-6359-4a5c-bea3-845c845ae6e9'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.110764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] The token cache is being invalidate because project 253f525611884991be947ca6c921f44a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:24.111059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.111724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.113950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1004/1984] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:23 2022] DELETE /identity/v3/projects/253f525611884991be947ca6c921f44a => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:24.115050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:24.116327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:24.176368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078b77e-6359-4a5c-bea3-845c845ae6e9 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:24.180038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cab7c7-6265-413f-aa0c-f0055e82a441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:24.180649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cab7c7-6265-413f-aa0c-f0055e82a441 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:24.180926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cab7c7-6265-413f-aa0c-f0055e82a441 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:24.181199 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cab7c7-6265-413f-aa0c-f0055e82a441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:24.183108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-70302b70-dc8f-474f-ba7b-1f1c48d62079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a683021de6904ca6b770d98a7051fb04'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.183256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-85cab7c7-6265-413f-aa0c-f0055e82a441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:24.184140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-70302b70-dc8f-474f-ba7b-1f1c48d62079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a683021de6904ca6b770d98a7051fb04'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.184719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-70302b70-dc8f-474f-ba7b-1f1c48d62079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a683021de6904ca6b770d98a7051fb04'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.194369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-70302b70-dc8f-474f-ba7b-1f1c48d62079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a683021de6904ca6b770d98a7051fb04'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.213216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 981/1985] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:23 2022] DELETE /identity/v3/users/a683021de6904ca6b770d98a7051fb04 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:24.214699 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a5e2f-75bb-418e-846e-70fe10f90e6d None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:24.215077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a5e2f-75bb-418e-846e-70fe10f90e6d None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:24.215531 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a5e2f-75bb-418e-846e-70fe10f90e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:24.235599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-663a5e2f-75bb-418e-846e-70fe10f90e6d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:24.236800 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-663a5e2f-75bb-418e-846e-70fe10f90e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:24.251799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cab7c7-6265-413f-aa0c-f0055e82a441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:24.257378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cab7c7-6265-413f-aa0c-f0055e82a441 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:24.264870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1005/1986] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:24.265778 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cab7c7-6265-413f-aa0c-f0055e82a441 service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:24.267143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-85cab7c7-6265-413f-aa0c-f0055e82a441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:24.286757 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cab7c7-6265-413f-aa0c-f0055e82a441 service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:24.317397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 982/1987] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:24.318775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce7afe-12f8-40d2-a568-283604cd9126 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:24.319067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce7afe-12f8-40d2-a568-283604cd9126 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:24.319347 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce7afe-12f8-40d2-a568-283604cd9126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:24.355191 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-37ce7afe-12f8-40d2-a568-283604cd9126 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:24.356368 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-37ce7afe-12f8-40d2-a568-283604cd9126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:24.365886 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 983/1988] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:24.369255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:24.369382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f978c2-d97d-4262-9c37-02fab1400126 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:24.369382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f978c2-d97d-4262-9c37-02fab1400126 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:24.369382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f978c2-d97d-4262-9c37-02fab1400126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:24.369654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:24.369654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:24.369654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] PATH_INFO: `/v3/projects/2cc9e578e09e465da03688f812b1a8c3` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:24.370707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc9e578e09e465da03688f812b1a8c3)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:24.372883 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:24.389415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-41f978c2-d97d-4262-9c37-02fab1400126 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:24.390698 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-41f978c2-d97d-4262-9c37-02fab1400126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:24.400300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 984/1989] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:24.401282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f978c2-d97d-4262-9c37-02fab1400126 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:24.402975 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-41f978c2-d97d-4262-9c37-02fab1400126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:24.464053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f978c2-d97d-4262-9c37-02fab1400126 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:24.468037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3013d2a-4e67-4c70-8fe6-4953be1d7c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:24.468758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3013d2a-4e67-4c70-8fe6-4953be1d7c88 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:24.469048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3013d2a-4e67-4c70-8fe6-4953be1d7c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:24.469328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3013d2a-4e67-4c70-8fe6-4953be1d7c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:24.479537 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3013d2a-4e67-4c70-8fe6-4953be1d7c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:24.480968 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3013d2a-4e67-4c70-8fe6-4953be1d7c88 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:24.491398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 985/1990] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:24.492307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3013d2a-4e67-4c70-8fe6-4953be1d7c88 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:24.501993 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3013d2a-4e67-4c70-8fe6-4953be1d7c88 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:24.509072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abc7fd-8def-4fcc-8ab0-968d24926119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:24.509464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abc7fd-8def-4fcc-8ab0-968d24926119 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:24.509756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abc7fd-8def-4fcc-8ab0-968d24926119 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:24.510037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abc7fd-8def-4fcc-8ab0-968d24926119 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:24.511063 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abc7fd-8def-4fcc-8ab0-968d24926119 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:24.511369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abc7fd-8def-4fcc-8ab0-968d24926119 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:24.513122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abc7fd-8def-4fcc-8ab0-968d24926119 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:24.531301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 986/1991] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:24.532482 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abc7fd-8def-4fcc-8ab0-968d24926119 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:24.536080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc9e578e09e465da03688f812b1a8c3', 'request_id': 'req-aad785a0-707c-4ccf-805d-70a305e997c8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.542978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abc7fd-8def-4fcc-8ab0-968d24926119 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:24.543167 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc9e578e09e465da03688f812b1a8c3', 'request_id': 'req-aad785a0-707c-4ccf-805d-70a305e997c8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.620008 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cf690-6b96-45d7-a480-71304dfa0e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:24.620633 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cf690-6b96-45d7-a480-71304dfa0e79 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:24.620921 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cf690-6b96-45d7-a480-71304dfa0e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:24.621198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cf690-6b96-45d7-a480-71304dfa0e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:24.621836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cf690-6b96-45d7-a480-71304dfa0e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:24.623884 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cf690-6b96-45d7-a480-71304dfa0e79 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:24.636791 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc9e578e09e465da03688f812b1a8c3', 'request_id': 'req-aad785a0-707c-4ccf-805d-70a305e997c8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.652866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] The token cache is being invalidate because project 2cc9e578e09e465da03688f812b1a8c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:24.653228 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.653969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.655876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1006/1992] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:24 2022] DELETE /identity/v3/projects/2cc9e578e09e465da03688f812b1a8c3 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:24.660363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:24.668482 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:24.675157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad785a0-707c-4ccf-805d-70a305e997c8 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:24.680133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:24.680879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:24.681221 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:24.681665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] PATH_INFO: `/v3/users/cb1971941f8944f88b968d9a78bba42a` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:24.682977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb1971941f8944f88b968d9a78bba42a)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:24.684938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:24.746467 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 987/1993] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:24 2022] POST /identity/v3/projects => generated 381 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:57:24.750867 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cf690-6b96-45d7-a480-71304dfa0e79 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:24.752879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-023cf690-6b96-45d7-a480-71304dfa0e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:24.826601 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cf690-6b96-45d7-a480-71304dfa0e79 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:24.835224 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535da5b-cf76-42fe-8e66-3b3bcd1999c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:24.835865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535da5b-cf76-42fe-8e66-3b3bcd1999c6 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:24.836144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535da5b-cf76-42fe-8e66-3b3bcd1999c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:24.836418 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535da5b-cf76-42fe-8e66-3b3bcd1999c6 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:24.837062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535da5b-cf76-42fe-8e66-3b3bcd1999c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:24.839110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535da5b-cf76-42fe-8e66-3b3bcd1999c6 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:24.935491 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 988/1994] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:24 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:57:24.937068 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535da5b-cf76-42fe-8e66-3b3bcd1999c6 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:24.938843 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c535da5b-cf76-42fe-8e66-3b3bcd1999c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:24.953670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535da5b-cf76-42fe-8e66-3b3bcd1999c6 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:24.963908 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:24.964536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb1971941f8944f88b968d9a78bba42a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:24.964680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:24.964844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:24.965115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] PATH_INFO: `/v3/projects/42d1aa5aaceb43aa868cdd6f560d75f2` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:24.966587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d1aa5aaceb43aa868cdd6f560d75f2)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:24.968502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:25.007802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb1971941f8944f88b968d9a78bba42a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.013242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb1971941f8944f88b968d9a78bba42a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.013982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb1971941f8944f88b968d9a78bba42a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.035898 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb1971941f8944f88b968d9a78bba42a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.061397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1007/1995] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:24 2022] DELETE /identity/v3/users/cb1971941f8944f88b968d9a78bba42a => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:25.062665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:25.066764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:25.088215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:25.156947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d1aa5aaceb43aa868cdd6f560d75f2', 'request_id': 'req-d9be562c-610b-458e-9767-60c45188b622'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.157155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d73806-479f-4416-8690-986c3d1bb4b6 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:25.168606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d1aa5aaceb43aa868cdd6f560d75f2', 'request_id': 'req-d9be562c-610b-458e-9767-60c45188b622'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.240556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb354b0-0d66-4e41-9a8a-edfecdbe2ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:25.244676 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb354b0-0d66-4e41-9a8a-edfecdbe2ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:25.244676 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb354b0-0d66-4e41-9a8a-edfecdbe2ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:25.244676 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb354b0-0d66-4e41-9a8a-edfecdbe2ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:25.244676 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb354b0-0d66-4e41-9a8a-edfecdbe2ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:25.245027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb354b0-0d66-4e41-9a8a-edfecdbe2ea5 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:25.257594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1008/1996] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:25.258606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb354b0-0d66-4e41-9a8a-edfecdbe2ea5 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:25.261017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fdb354b0-0d66-4e41-9a8a-edfecdbe2ea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:25.264756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d1aa5aaceb43aa868cdd6f560d75f2', 'request_id': 'req-d9be562c-610b-458e-9767-60c45188b622'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.276429 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] The token cache is being invalidate because project 42d1aa5aaceb43aa868cdd6f560d75f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:25.276796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.278670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.279368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb354b0-0d66-4e41-9a8a-edfecdbe2ea5 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:25.280746 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 989/1997] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:24 2022] DELETE /identity/v3/projects/42d1aa5aaceb43aa868cdd6f560d75f2 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:25.282268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:25.285589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:25.287297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:25.288862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:25.289305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:25.289709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] PATH_INFO: `/v3/projects/e7b789c30cf647c69e1a59bf3b99eda3` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:25.294834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7b789c30cf647c69e1a59bf3b99eda3)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:25.296786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:25.301900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be562c-610b-458e-9767-60c45188b622 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:25.308773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83923f2-df62-4125-a1ff-a68351579034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:25.309728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83923f2-df62-4125-a1ff-a68351579034 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:25.310016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83923f2-df62-4125-a1ff-a68351579034 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:25.310293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83923f2-df62-4125-a1ff-a68351579034 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:25.311057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83923f2-df62-4125-a1ff-a68351579034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:25.312678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83923f2-df62-4125-a1ff-a68351579034 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:25.330109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 990/1998] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:25.331295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83923f2-df62-4125-a1ff-a68351579034 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:25.334152 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c83923f2-df62-4125-a1ff-a68351579034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:25.346038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83923f2-df62-4125-a1ff-a68351579034 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:25.355109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:25.356287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:25.356822 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:25.357344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] PATH_INFO: `/v3/users/f55189270fbf4eb58cf2124c2ecc8c61` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:25.360153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f55189270fbf4eb58cf2124c2ecc8c61)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:25.365258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:25.446445 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7b789c30cf647c69e1a59bf3b99eda3', 'request_id': 'req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.455925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7b789c30cf647c69e1a59bf3b99eda3', 'request_id': 'req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.458077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:25.516584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7b789c30cf647c69e1a59bf3b99eda3', 'request_id': 'req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.530113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] The token cache is being invalidate because project e7b789c30cf647c69e1a59bf3b99eda3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:25.530407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.531134 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.532798 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1009/1999] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:25 2022] DELETE /identity/v3/projects/e7b789c30cf647c69e1a59bf3b99eda3 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:25.539157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:25.539157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:25.601472 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f55189270fbf4eb58cf2124c2ecc8c61'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.623016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9668a-ab5f-450d-9a24-4cc3b9c16b48 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:25.629911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cff107-ccb8-444d-9e4b-2b3a4cabf0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:25.629911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cff107-ccb8-444d-9e4b-2b3a4cabf0ce admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:25.629911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cff107-ccb8-444d-9e4b-2b3a4cabf0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:25.629911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cff107-ccb8-444d-9e4b-2b3a4cabf0ce admin admin] PATH_INFO: `/v3/projects/d0bcf4149ec14ca3abbde516fe1b807a/users/ebb96d98580040ac88d469bbb2f313b5/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:25.646153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cff107-ccb8-444d-9e4b-2b3a4cabf0ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0bcf4149ec14ca3abbde516fe1b807a, user_id=ebb96d98580040ac88d469bbb2f313b5, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:25.649037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cff107-ccb8-444d-9e4b-2b3a4cabf0ce admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:25.697385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f55189270fbf4eb58cf2124c2ecc8c61'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.698011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f55189270fbf4eb58cf2124c2ecc8c61'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.698721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f55189270fbf4eb58cf2124c2ecc8c61'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.710631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f55189270fbf4eb58cf2124c2ecc8c61'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:25.734797 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1010/2000] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:25 2022] PUT /identity/v3/projects/d0bcf4149ec14ca3abbde516fe1b807a/users/ebb96d98580040ac88d469bbb2f313b5/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:25.735660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cff107-ccb8-444d-9e4b-2b3a4cabf0ce admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:25.736779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 991/2001] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:25 2022] DELETE /identity/v3/users/f55189270fbf4eb58cf2124c2ecc8c61 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:25.737743 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:25.739742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:25.742410 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-76cff107-ccb8-444d-9e4b-2b3a4cabf0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:25.834300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cff107-ccb8-444d-9e4b-2b3a4cabf0ce admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:25.877297 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ef74-e512-434c-a291-86344e1de6f4 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:25.887099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:25.888447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:25.908464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:25.908464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] PATH_INFO: `/v3/projects/d804ababca4e48039d7daa25b6de4a41` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:25.908464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d804ababca4e48039d7daa25b6de4a41)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:25.908464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:25.920717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75039ee-be44-4346-b586-ab1718d5ab14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:25.921359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75039ee-be44-4346-b586-ab1718d5ab14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:25.921816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75039ee-be44-4346-b586-ab1718d5ab14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:25.922094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75039ee-be44-4346-b586-ab1718d5ab14 admin admin] PATH_INFO: `/v3/users/6b54198943de4072991f0e2feca855fa` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:25.923542 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75039ee-be44-4346-b586-ab1718d5ab14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b54198943de4072991f0e2feca855fa)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:25.926353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75039ee-be44-4346-b586-ab1718d5ab14 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:26.141806 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b75039ee-be44-4346-b586-ab1718d5ab14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b54198943de4072991f0e2feca855fa'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.142038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd804ababca4e48039d7daa25b6de4a41', 'request_id': 'req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.149918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd804ababca4e48039d7daa25b6de4a41', 'request_id': 'req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.230378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b75039ee-be44-4346-b586-ab1718d5ab14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b54198943de4072991f0e2feca855fa'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.231191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b75039ee-be44-4346-b586-ab1718d5ab14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b54198943de4072991f0e2feca855fa'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.231974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b75039ee-be44-4346-b586-ab1718d5ab14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b54198943de4072991f0e2feca855fa'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.245391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b75039ee-be44-4346-b586-ab1718d5ab14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b54198943de4072991f0e2feca855fa'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.258045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1011/2002] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:25 2022] DELETE /identity/v3/users/6b54198943de4072991f0e2feca855fa => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:26.259154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a909d4e-4024-4310-80df-5129a67a452f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:26.259435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a909d4e-4024-4310-80df-5129a67a452f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:26.259715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a909d4e-4024-4310-80df-5129a67a452f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:26.291611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-1a909d4e-4024-4310-80df-5129a67a452f None None] MFA Rules not processed for user `ebb96d98580040ac88d469bbb2f313b5`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:26.292840 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1a909d4e-4024-4310-80df-5129a67a452f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:26.308835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd804ababca4e48039d7daa25b6de4a41', 'request_id': 'req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.322132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] The token cache is being invalidate because project d804ababca4e48039d7daa25b6de4a41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:26.322852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.324020 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.327794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 992/2003] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:25 2022] DELETE /identity/v3/projects/d804ababca4e48039d7daa25b6de4a41 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:26.329829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:26.331460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:26.377005 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1012/2004] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:26.378405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b6b7b-676b-4f5f-b4b9-393af78cd7a9 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:26.378981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b6b7b-676b-4f5f-b4b9-393af78cd7a9 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:26.379326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b6b7b-676b-4f5f-b4b9-393af78cd7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:26.396776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:26.400305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d83b6b7b-676b-4f5f-b4b9-393af78cd7a9 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:26.404895 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d83b6b7b-676b-4f5f-b4b9-393af78cd7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:26.408455 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0931b0-c7d5-4ae0-88a7-108fa57f1ec2 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:26.468766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1013/2005] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:26.468766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b6b7b-676b-4f5f-b4b9-393af78cd7a9 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:26.470159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d83b6b7b-676b-4f5f-b4b9-393af78cd7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:26.484161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b6b7b-676b-4f5f-b4b9-393af78cd7a9 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:26.490339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170a1eb-1139-45c7-a2f9-0944c6262cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:26.491196 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170a1eb-1139-45c7-a2f9-0944c6262cbe admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:26.491493 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170a1eb-1139-45c7-a2f9-0944c6262cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:26.491768 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170a1eb-1139-45c7-a2f9-0944c6262cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:26.492469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170a1eb-1139-45c7-a2f9-0944c6262cbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:26.494696 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170a1eb-1139-45c7-a2f9-0944c6262cbe admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:26.536540 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:26.537295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:26.538913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:26.539276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] PATH_INFO: `/v3/projects/6e827c84813c4d0a8d3eff37ec8ddf81` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:26.540548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e827c84813c4d0a8d3eff37ec8ddf81)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:26.542563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:26.615798 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1014/2006] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:26 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:26.617283 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bcaec6-a64b-48f1-bea1-35f076f2e92a None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:26.617711 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bcaec6-a64b-48f1-bea1-35f076f2e92a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:26.618163 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bcaec6-a64b-48f1-bea1-35f076f2e92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:26.638849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-29bcaec6-a64b-48f1-bea1-35f076f2e92a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:26.640174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-29bcaec6-a64b-48f1-bea1-35f076f2e92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:26.657763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e827c84813c4d0a8d3eff37ec8ddf81', 'request_id': 'req-3aa4cb93-9192-462a-b407-d3fad943c6b2'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.664529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e827c84813c4d0a8d3eff37ec8ddf81', 'request_id': 'req-3aa4cb93-9192-462a-b407-d3fad943c6b2'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.716845 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1015/2007] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:26.720306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bcaec6-a64b-48f1-bea1-35f076f2e92a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:26.727573 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bcaec6-a64b-48f1-bea1-35f076f2e92a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:26.741567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e827c84813c4d0a8d3eff37ec8ddf81', 'request_id': 'req-3aa4cb93-9192-462a-b407-d3fad943c6b2'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.756013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] The token cache is being invalidate because project 6e827c84813c4d0a8d3eff37ec8ddf81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:26.756317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.756985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.758534 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 993/2008] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:26 2022] DELETE /identity/v3/projects/6e827c84813c4d0a8d3eff37ec8ddf81 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:26.759304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:26.760638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:26.789373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fb56b-bec6-4b07-b542-04ca0d29a835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:26.789989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fb56b-bec6-4b07-b542-04ca0d29a835 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:26.790428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fb56b-bec6-4b07-b542-04ca0d29a835 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:26.790738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fb56b-bec6-4b07-b542-04ca0d29a835 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:26.791378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fb56b-bec6-4b07-b542-04ca0d29a835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:26.793771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fb56b-bec6-4b07-b542-04ca0d29a835 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:26.828304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4cb93-9192-462a-b407-d3fad943c6b2 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:26.834244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64301e1a-bb1f-40f2-971c-46c3057d6d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:26.834863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64301e1a-bb1f-40f2-971c-46c3057d6d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:26.835145 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64301e1a-bb1f-40f2-971c-46c3057d6d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:26.835423 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64301e1a-bb1f-40f2-971c-46c3057d6d1e admin admin] PATH_INFO: `/v3/projects/a0892bd82d6346f5b44e663d73692e2a` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:26.836581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64301e1a-bb1f-40f2-971c-46c3057d6d1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0892bd82d6346f5b44e663d73692e2a)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:26.838474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64301e1a-bb1f-40f2-971c-46c3057d6d1e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:26.858870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1016/2009] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:26 2022] POST /identity/v3/users => generated 297 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:26.860059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fb56b-bec6-4b07-b542-04ca0d29a835 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:26.863366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fd5fb56b-bec6-4b07-b542-04ca0d29a835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:26.874986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fb56b-bec6-4b07-b542-04ca0d29a835 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:26.880041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b7df6-bed0-4439-a8b1-68b168f64aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:26.881784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b7df6-bed0-4439-a8b1-68b168f64aeb admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:26.881784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b7df6-bed0-4439-a8b1-68b168f64aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:26.881784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b7df6-bed0-4439-a8b1-68b168f64aeb admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:26.881784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b7df6-bed0-4439-a8b1-68b168f64aeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:26.882132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b7df6-bed0-4439-a8b1-68b168f64aeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:26.883667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b7df6-bed0-4439-a8b1-68b168f64aeb admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:26.898527 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1017/2010] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:26.899007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afd3df-eed9-47a9-a69e-c3857a2185b1 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:26.899275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afd3df-eed9-47a9-a69e-c3857a2185b1 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:26.899556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afd3df-eed9-47a9-a69e-c3857a2185b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:26.919085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b5afd3df-eed9-47a9-a69e-c3857a2185b1 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:26.920294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b5afd3df-eed9-47a9-a69e-c3857a2185b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:26.929463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1018/2011] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:26.930149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afd3df-eed9-47a9-a69e-c3857a2185b1 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:26.936966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b5afd3df-eed9-47a9-a69e-c3857a2185b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:26.950188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afd3df-eed9-47a9-a69e-c3857a2185b1 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:26.950996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-64301e1a-bb1f-40f2-971c-46c3057d6d1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0892bd82d6346f5b44e663d73692e2a', 'request_id': 'req-64301e1a-bb1f-40f2-971c-46c3057d6d1e'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:26.958549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-64301e1a-bb1f-40f2-971c-46c3057d6d1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0892bd82d6346f5b44e663d73692e2a', 'request_id': 'req-64301e1a-bb1f-40f2-971c-46c3057d6d1e'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:27.007704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655915d-e0ea-4d5b-a518-35f88112cf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.008302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655915d-e0ea-4d5b-a518-35f88112cf64 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.008576 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655915d-e0ea-4d5b-a518-35f88112cf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.008841 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655915d-e0ea-4d5b-a518-35f88112cf64 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.009508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655915d-e0ea-4d5b-a518-35f88112cf64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.011174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655915d-e0ea-4d5b-a518-35f88112cf64 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.020866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1019/2012] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:27.021807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655915d-e0ea-4d5b-a518-35f88112cf64 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.025626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-64301e1a-bb1f-40f2-971c-46c3057d6d1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0892bd82d6346f5b44e663d73692e2a', 'request_id': 'req-64301e1a-bb1f-40f2-971c-46c3057d6d1e'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:27.031322 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655915d-e0ea-4d5b-a518-35f88112cf64 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.037571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af31b0e-3dba-4a10-ad15-bdce30230dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.037752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-64301e1a-bb1f-40f2-971c-46c3057d6d1e admin admin] The token cache is being invalidate because project a0892bd82d6346f5b44e663d73692e2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:27.038019 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-64301e1a-bb1f-40f2-971c-46c3057d6d1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:27.038306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af31b0e-3dba-4a10-ad15-bdce30230dca admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.038721 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af31b0e-3dba-4a10-ad15-bdce30230dca admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.038842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-64301e1a-bb1f-40f2-971c-46c3057d6d1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:27.038991 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af31b0e-3dba-4a10-ad15-bdce30230dca admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.039614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af31b0e-3dba-4a10-ad15-bdce30230dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.040383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 994/2013] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:26 2022] DELETE /identity/v3/projects/a0892bd82d6346f5b44e663d73692e2a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:27.041551 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af31b0e-3dba-4a10-ad15-bdce30230dca admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.041672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4cb62-ba89-43f6-a597-fc622d09afe1 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.041963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4cb62-ba89-43f6-a597-fc622d09afe1 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.042242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4cb62-ba89-43f6-a597-fc622d09afe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.064076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-23d4cb62-ba89-43f6-a597-fc622d09afe1 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:27.065268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-23d4cb62-ba89-43f6-a597-fc622d09afe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:27.074596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 995/2014] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:27.075739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4cb62-ba89-43f6-a597-fc622d09afe1 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.077132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-23d4cb62-ba89-43f6-a597-fc622d09afe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:27.095847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4cb62-ba89-43f6-a597-fc622d09afe1 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.110261 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3cc4ef-e70f-47e7-8290-d058131a6f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.110973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3cc4ef-e70f-47e7-8290-d058131a6f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.111252 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3cc4ef-e70f-47e7-8290-d058131a6f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.111523 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3cc4ef-e70f-47e7-8290-d058131a6f6f admin admin] PATH_INFO: `/v3/projects/9f48d391ca7c438b94c9ba699c0eb724/users/7bba8cd408aa4a219fe35c8380ba17ed/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.132102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3cc4ef-e70f-47e7-8290-d058131a6f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f48d391ca7c438b94c9ba699c0eb724, user_id=7bba8cd408aa4a219fe35c8380ba17ed, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.134855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3cc4ef-e70f-47e7-8290-d058131a6f6f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.145071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1020/2015] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:27 2022] POST /identity/v3/projects => generated 389 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:27.147397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc10e76-7249-45c2-8c59-c5452907a17e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.147397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc10e76-7249-45c2-8c59-c5452907a17e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.147397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc10e76-7249-45c2-8c59-c5452907a17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.165729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-ffc10e76-7249-45c2-8c59-c5452907a17e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:27.167038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ffc10e76-7249-45c2-8c59-c5452907a17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:27.227503 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1021/2016] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:27.228371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc10e76-7249-45c2-8c59-c5452907a17e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.237715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc10e76-7249-45c2-8c59-c5452907a17e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.243423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e76e7-95b1-4cd6-9b1e-b417f0b09028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.244019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e76e7-95b1-4cd6-9b1e-b417f0b09028 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.244301 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e76e7-95b1-4cd6-9b1e-b417f0b09028 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.244576 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e76e7-95b1-4cd6-9b1e-b417f0b09028 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.245222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e76e7-95b1-4cd6-9b1e-b417f0b09028 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:27.245526 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e76e7-95b1-4cd6-9b1e-b417f0b09028 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.247146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e76e7-95b1-4cd6-9b1e-b417f0b09028 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.258742 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1022/2017] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:27.259522 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e76e7-95b1-4cd6-9b1e-b417f0b09028 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.261175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9f2e76e7-95b1-4cd6-9b1e-b417f0b09028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:27.275097 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e76e7-95b1-4cd6-9b1e-b417f0b09028 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.279673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37870c2d-0d19-4c15-81ca-bfa82af18c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.280273 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37870c2d-0d19-4c15-81ca-bfa82af18c5a admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.280551 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37870c2d-0d19-4c15-81ca-bfa82af18c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.280843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37870c2d-0d19-4c15-81ca-bfa82af18c5a admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.281457 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37870c2d-0d19-4c15-81ca-bfa82af18c5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.283340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37870c2d-0d19-4c15-81ca-bfa82af18c5a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.294574 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 996/2018] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:27 2022] PUT /identity/v3/projects/9f48d391ca7c438b94c9ba699c0eb724/users/7bba8cd408aa4a219fe35c8380ba17ed/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:27.295894 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedeede1-1029-4eb9-a764-600395707896 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.296239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedeede1-1029-4eb9-a764-600395707896 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.296522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedeede1-1029-4eb9-a764-600395707896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.315541 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-cedeede1-1029-4eb9-a764-600395707896 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:27.316971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-cedeede1-1029-4eb9-a764-600395707896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:27.351489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:27.389986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1023/2019] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:27 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:27.389986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37870c2d-0d19-4c15-81ca-bfa82af18c5a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.392066 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 997/2020] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:27.392996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedeede1-1029-4eb9-a764-600395707896 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.399956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37870c2d-0d19-4c15-81ca-bfa82af18c5a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.403284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedeede1-1029-4eb9-a764-600395707896 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.404708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53702957-4a81-4290-b28d-0008043a26c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.405312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53702957-4a81-4290-b28d-0008043a26c6 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.405589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53702957-4a81-4290-b28d-0008043a26c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.405858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53702957-4a81-4290-b28d-0008043a26c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.406511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53702957-4a81-4290-b28d-0008043a26c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.408390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53702957-4a81-4290-b28d-0008043a26c6 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.408624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ea64b-7622-4316-827f-447c85c6a9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.409219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ea64b-7622-4316-827f-447c85c6a9ce admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.411770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ea64b-7622-4316-827f-447c85c6a9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.411770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ea64b-7622-4316-827f-447c85c6a9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.411770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ea64b-7622-4316-827f-447c85c6a9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.412613 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ea64b-7622-4316-827f-447c85c6a9ce admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.423081 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 998/2021] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:27.424113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ea64b-7622-4316-827f-447c85c6a9ce admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.434095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ea64b-7622-4316-827f-447c85c6a9ce admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.440170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b67964-f645-441a-810a-036c30c1732f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.441727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b67964-f645-441a-810a-036c30c1732f admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.441727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b67964-f645-441a-810a-036c30c1732f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.441727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b67964-f645-441a-810a-036c30c1732f admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.442315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b67964-f645-441a-810a-036c30c1732f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.444174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b67964-f645-441a-810a-036c30c1732f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.455379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 999/2022] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:27.456344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b67964-f645-441a-810a-036c30c1732f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.468806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b67964-f645-441a-810a-036c30c1732f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.474297 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8990741-e6ee-4009-b5f7-cd2c36d227d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.474965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8990741-e6ee-4009-b5f7-cd2c36d227d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.475244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8990741-e6ee-4009-b5f7-cd2c36d227d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.475515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8990741-e6ee-4009-b5f7-cd2c36d227d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.476152 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8990741-e6ee-4009-b5f7-cd2c36d227d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:27.476482 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8990741-e6ee-4009-b5f7-cd2c36d227d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.478102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8990741-e6ee-4009-b5f7-cd2c36d227d4 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.494673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1000/2023] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:27.495462 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1024/2024] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:27 2022] POST /identity/v3/projects => generated 399 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:27.495629 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8990741-e6ee-4009-b5f7-cd2c36d227d4 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.496734 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53702957-4a81-4290-b28d-0008043a26c6 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.509514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8990741-e6ee-4009-b5f7-cd2c36d227d4 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.591049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81ed57-7979-4a69-a476-7ede92666ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.591753 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81ed57-7979-4a69-a476-7ede92666ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.592035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81ed57-7979-4a69-a476-7ede92666ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.592310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81ed57-7979-4a69-a476-7ede92666ffc admin admin] PATH_INFO: `/v3/projects/9f48d391ca7c438b94c9ba699c0eb724/users/7bba8cd408aa4a219fe35c8380ba17ed/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.594514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81ed57-7979-4a69-a476-7ede92666ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f48d391ca7c438b94c9ba699c0eb724, user_id=7bba8cd408aa4a219fe35c8380ba17ed, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.594888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53702957-4a81-4290-b28d-0008043a26c6 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.597107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81ed57-7979-4a69-a476-7ede92666ffc admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.600949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e7c7d-6e44-41cc-8340-a000bd4eabfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.601706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e7c7d-6e44-41cc-8340-a000bd4eabfb admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.605528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e7c7d-6e44-41cc-8340-a000bd4eabfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.605528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e7c7d-6e44-41cc-8340-a000bd4eabfb admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.605528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e7c7d-6e44-41cc-8340-a000bd4eabfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.605528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e7c7d-6e44-41cc-8340-a000bd4eabfb admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.608715 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1001/2025] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:27 2022] PUT /identity/v3/projects/9f48d391ca7c438b94c9ba699c0eb724/users/7bba8cd408aa4a219fe35c8380ba17ed/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:27.609502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81ed57-7979-4a69-a476-7ede92666ffc admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.622907 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1025/2026] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:27.623767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e7c7d-6e44-41cc-8340-a000bd4eabfb admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.633755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e7c7d-6e44-41cc-8340-a000bd4eabfb admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.683364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81ed57-7979-4a69-a476-7ede92666ffc admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.689363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2521f0-19cc-46f3-95d0-8011d4927711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.692712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2521f0-19cc-46f3-95d0-8011d4927711 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.692712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2521f0-19cc-46f3-95d0-8011d4927711 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.692712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2521f0-19cc-46f3-95d0-8011d4927711 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.692712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2521f0-19cc-46f3-95d0-8011d4927711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.694270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2521f0-19cc-46f3-95d0-8011d4927711 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.701844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0435d1-1849-479d-b659-b3ea8d7855c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.702474 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0435d1-1849-479d-b659-b3ea8d7855c3 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.703114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0435d1-1849-479d-b659-b3ea8d7855c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.703389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0435d1-1849-479d-b659-b3ea8d7855c3 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.704013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0435d1-1849-479d-b659-b3ea8d7855c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.705929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0435d1-1849-479d-b659-b3ea8d7855c3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.796226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1026/2027] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:27 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:27.800469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1002/2028] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:27 2022] POST /identity/v3/projects => generated 387 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:27.800469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ecbf7-5995-48b3-a449-d2ee74065d87 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.800469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ecbf7-5995-48b3-a449-d2ee74065d87 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.800469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ecbf7-5995-48b3-a449-d2ee74065d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.800850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0435d1-1849-479d-b659-b3ea8d7855c3 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.810675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0435d1-1849-479d-b659-b3ea8d7855c3 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.826970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-9c8ecbf7-5995-48b3-a449-d2ee74065d87 None None] MFA Rules not processed for user `7bba8cd408aa4a219fe35c8380ba17ed`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:27.828758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9c8ecbf7-5995-48b3-a449-d2ee74065d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:27.850549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:27.875501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec641c-a75d-42e4-a8a5-5f5e00498624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.876179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec641c-a75d-42e4-a8a5-5f5e00498624 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.876459 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec641c-a75d-42e4-a8a5-5f5e00498624 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.876736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec641c-a75d-42e4-a8a5-5f5e00498624 admin admin] PATH_INFO: `/v3/projects/2630da85cb8d4f0a9d01897f20152567/users/b459c5ca6d524f5a9de282f8fc975a1a/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.899734 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec641c-a75d-42e4-a8a5-5f5e00498624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2630da85cb8d4f0a9d01897f20152567, user_id=b459c5ca6d524f5a9de282f8fc975a1a, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.899734 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec641c-a75d-42e4-a8a5-5f5e00498624 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.906257 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1003/2029] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:27.907109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ecbf7-5995-48b3-a449-d2ee74065d87 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.917270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ecbf7-5995-48b3-a449-d2ee74065d87 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.921956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5be9c-5340-44bf-8f36-998b94934a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.922586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5be9c-5340-44bf-8f36-998b94934a16 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.922869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5be9c-5340-44bf-8f36-998b94934a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.923140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5be9c-5340-44bf-8f36-998b94934a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.923768 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5be9c-5340-44bf-8f36-998b94934a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.925455 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5be9c-5340-44bf-8f36-998b94934a16 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.936059 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1004/2030] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:27.936854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5be9c-5340-44bf-8f36-998b94934a16 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.946636 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5be9c-5340-44bf-8f36-998b94934a16 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:27.951230 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5b01e-ced6-4171-b466-de81f25fb800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:27.951806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5b01e-ced6-4171-b466-de81f25fb800 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:27.952097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5b01e-ced6-4171-b466-de81f25fb800 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:27.952351 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5b01e-ced6-4171-b466-de81f25fb800 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:27.953227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5b01e-ced6-4171-b466-de81f25fb800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:27.955184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5b01e-ced6-4171-b466-de81f25fb800 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:27.968246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1027/2031] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:27 2022] PUT /identity/v3/projects/2630da85cb8d4f0a9d01897f20152567/users/b459c5ca6d524f5a9de282f8fc975a1a/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:27.969555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec641c-a75d-42e4-a8a5-5f5e00498624 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:27.971911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:27.974092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-10ec641c-a75d-42e4-a8a5-5f5e00498624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:28.030697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec641c-a75d-42e4-a8a5-5f5e00498624 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:28.034316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649debe2-ae31-4c48-a3b4-ee7d38ad3c4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:28.034940 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649debe2-ae31-4c48-a3b4-ee7d38ad3c4e service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.035214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649debe2-ae31-4c48-a3b4-ee7d38ad3c4e service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.035605 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649debe2-ae31-4c48-a3b4-ee7d38ad3c4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.096430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1005/2032] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:27 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:28.096430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5b01e-ced6-4171-b466-de81f25fb800 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:28.099067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649debe2-ae31-4c48-a3b4-ee7d38ad3c4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:28.100549 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649debe2-ae31-4c48-a3b4-ee7d38ad3c4e service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:28.104166 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5b01e-ced6-4171-b466-de81f25fb800 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:28.110857 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1028/2033] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:28.112344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5393f82-1fe0-4d4b-ae19-3522a0c0e88d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.112634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5393f82-1fe0-4d4b-ae19-3522a0c0e88d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.112911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5393f82-1fe0-4d4b-ae19-3522a0c0e88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.143761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-c5393f82-1fe0-4d4b-ae19-3522a0c0e88d None None] MFA Rules not processed for user `b459c5ca6d524f5a9de282f8fc975a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:28.145049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c5393f82-1fe0-4d4b-ae19-3522a0c0e88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:28.173779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0157e3-b0ff-42e6-8a6c-fe3e8e7c7eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:28.174469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0157e3-b0ff-42e6-8a6c-fe3e8e7c7eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.174786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0157e3-b0ff-42e6-8a6c-fe3e8e7c7eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.175204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0157e3-b0ff-42e6-8a6c-fe3e8e7c7eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.175859 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0157e3-b0ff-42e6-8a6c-fe3e8e7c7eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:28.178833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0157e3-b0ff-42e6-8a6c-fe3e8e7c7eb9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:28.193787 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1006/2034] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:28.193787 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0157e3-b0ff-42e6-8a6c-fe3e8e7c7eb9 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:28.201902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0157e3-b0ff-42e6-8a6c-fe3e8e7c7eb9 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:28.207073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ea93a-c464-4f00-a9d3-bf7826f93382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:28.208700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ea93a-c464-4f00-a9d3-bf7826f93382 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.208981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ea93a-c464-4f00-a9d3-bf7826f93382 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.209398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ea93a-c464-4f00-a9d3-bf7826f93382 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.210010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ea93a-c464-4f00-a9d3-bf7826f93382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:28.211835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ea93a-c464-4f00-a9d3-bf7826f93382 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:28.225485 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1007/2035] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:28.230646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ea93a-c464-4f00-a9d3-bf7826f93382 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:28.239305 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ea93a-c464-4f00-a9d3-bf7826f93382 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:28.244065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6865b4-c6d1-4f74-bfa6-c6c7b62a1a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:28.244728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6865b4-c6d1-4f74-bfa6-c6c7b62a1a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.245039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6865b4-c6d1-4f74-bfa6-c6c7b62a1a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.245335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6865b4-c6d1-4f74-bfa6-c6c7b62a1a35 admin admin] PATH_INFO: `/v3/projects/309d94b39e8c403eb5f36cf2ccd9c55d/users/9e642f25cca546f996d23fb579147c7b/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.249999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1029/2036] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:28.251008 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5393f82-1fe0-4d4b-ae19-3522a0c0e88d None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:28.262353 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6865b4-c6d1-4f74-bfa6-c6c7b62a1a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=309d94b39e8c403eb5f36cf2ccd9c55d, user_id=9e642f25cca546f996d23fb579147c7b, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:28.263947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5393f82-1fe0-4d4b-ae19-3522a0c0e88d None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:28.265216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6865b4-c6d1-4f74-bfa6-c6c7b62a1a35 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:28.270017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f5ec7-2fd0-4239-b1a9-1e1d44e72e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:28.270844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f5ec7-2fd0-4239-b1a9-1e1d44e72e22 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.271197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f5ec7-2fd0-4239-b1a9-1e1d44e72e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.271537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f5ec7-2fd0-4239-b1a9-1e1d44e72e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.272498 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f5ec7-2fd0-4239-b1a9-1e1d44e72e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:28.274197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f5ec7-2fd0-4239-b1a9-1e1d44e72e22 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:28.284730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1030/2037] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:28.286168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f5ec7-2fd0-4239-b1a9-1e1d44e72e22 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:28.297376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f5ec7-2fd0-4239-b1a9-1e1d44e72e22 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:28.302448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46c023-5239-4aae-b7e1-ac17c25421ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:28.303188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46c023-5239-4aae-b7e1-ac17c25421ce admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.303473 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46c023-5239-4aae-b7e1-ac17c25421ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.303747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46c023-5239-4aae-b7e1-ac17c25421ce admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.304383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46c023-5239-4aae-b7e1-ac17c25421ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:28.306268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46c023-5239-4aae-b7e1-ac17c25421ce admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:28.320755 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1008/2038] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:28 2022] PUT /identity/v3/projects/309d94b39e8c403eb5f36cf2ccd9c55d/users/9e642f25cca546f996d23fb579147c7b/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:28.321759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6865b4-c6d1-4f74-bfa6-c6c7b62a1a35 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:28.384595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1031/2039] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:28 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:28.384595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5e97c-f560-4555-9e5e-4be179e78707 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.384595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5e97c-f560-4555-9e5e-4be179e78707 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.384595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5e97c-f560-4555-9e5e-4be179e78707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.388806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:28.392719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6865b4-c6d1-4f74-bfa6-c6c7b62a1a35 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:28.416711 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-aaf5e97c-f560-4555-9e5e-4be179e78707 None None] MFA Rules not processed for user `9e642f25cca546f996d23fb579147c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:28.418011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-aaf5e97c-f560-4555-9e5e-4be179e78707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:28.456278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af9024-b4ec-4bc8-a808-56fae5becee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:28.457026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af9024-b4ec-4bc8-a808-56fae5becee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.457390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af9024-b4ec-4bc8-a808-56fae5becee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.457662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af9024-b4ec-4bc8-a808-56fae5becee4 admin admin] PATH_INFO: `/v3/projects/aea3816cd39d4bcda0818af6e505a207/users/5faa1995e8d34499bc63b2564e30758c/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.473634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af9024-b4ec-4bc8-a808-56fae5becee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea3816cd39d4bcda0818af6e505a207, user_id=5faa1995e8d34499bc63b2564e30758c, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:28.476779 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af9024-b4ec-4bc8-a808-56fae5becee4 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:28.496325 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1032/2040] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:28 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:28.497380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5e97c-f560-4555-9e5e-4be179e78707 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:28.508388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5e97c-f560-4555-9e5e-4be179e78707 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:28.514107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a571f-07b5-4bf8-af01-d0d150eea2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:28.514693 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a571f-07b5-4bf8-af01-d0d150eea2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.514971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a571f-07b5-4bf8-af01-d0d150eea2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.515243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a571f-07b5-4bf8-af01-d0d150eea2e9 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.517024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a571f-07b5-4bf8-af01-d0d150eea2e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:28.517792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a571f-07b5-4bf8-af01-d0d150eea2e9 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:28.581008 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1009/2041] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:28 2022] PUT /identity/v3/projects/aea3816cd39d4bcda0818af6e505a207/users/5faa1995e8d34499bc63b2564e30758c/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:28.581914 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af9024-b4ec-4bc8-a808-56fae5becee4 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:28.611379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:28.622588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1033/2042] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:28 2022] POST /identity/v3/users => generated 300 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:28.624321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67093781-1fe0-442d-acfb-789eebb52b1e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.624463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67093781-1fe0-442d-acfb-789eebb52b1e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.624797 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67093781-1fe0-442d-acfb-789eebb52b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.656537 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af9024-b4ec-4bc8-a808-56fae5becee4 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:28.658010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-67093781-1fe0-442d-acfb-789eebb52b1e None None] MFA Rules not processed for user `5faa1995e8d34499bc63b2564e30758c`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:28.659274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-67093781-1fe0-442d-acfb-789eebb52b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:28.662116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d309f-c5b6-449c-b6ec-712b1c413e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:28.662875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d309f-c5b6-449c-b6ec-712b1c413e82 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.663252 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d309f-c5b6-449c-b6ec-712b1c413e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.663836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d309f-c5b6-449c-b6ec-712b1c413e82 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.664553 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d309f-c5b6-449c-b6ec-712b1c413e82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:28.666587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d309f-c5b6-449c-b6ec-712b1c413e82 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:28.717010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:28.731112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1034/2043] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:28.732099 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67093781-1fe0-442d-acfb-789eebb52b1e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:28.745710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67093781-1fe0-442d-acfb-789eebb52b1e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:28.751328 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b982df7-aff8-4ded-bcfe-570c4cf015d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:28.755226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b982df7-aff8-4ded-bcfe-570c4cf015d7 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.755226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b982df7-aff8-4ded-bcfe-570c4cf015d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.755226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b982df7-aff8-4ded-bcfe-570c4cf015d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.755226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b982df7-aff8-4ded-bcfe-570c4cf015d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:28.755982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b982df7-aff8-4ded-bcfe-570c4cf015d7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:28.770770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1035/2044] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:28.770770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b982df7-aff8-4ded-bcfe-570c4cf015d7 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:28.779716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b982df7-aff8-4ded-bcfe-570c4cf015d7 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:28.783981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1010/2045] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:28 2022] POST /identity/v3/projects => generated 399 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:28.783981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d309f-c5b6-449c-b6ec-712b1c413e82 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:28.845334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b53265-e9d0-4345-bb44-947da71fbf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:28.845942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b53265-e9d0-4345-bb44-947da71fbf86 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.846217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b53265-e9d0-4345-bb44-947da71fbf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.846508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b53265-e9d0-4345-bb44-947da71fbf86 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.847145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b53265-e9d0-4345-bb44-947da71fbf86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:28.849548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b53265-e9d0-4345-bb44-947da71fbf86 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:28.858618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d309f-c5b6-449c-b6ec-712b1c413e82 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:28.863290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef15fa-7558-414a-903b-2a30bf363bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:28.864479 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef15fa-7558-414a-903b-2a30bf363bda admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:28.864941 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef15fa-7558-414a-903b-2a30bf363bda admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:28.865263 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef15fa-7558-414a-903b-2a30bf363bda admin admin] PATH_INFO: `/v3/projects/ceba0fd5aef942ca97a15cf52258ae16/users/78cc9bddddbd4111ba0cfdf0cd9ed12f/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:28.880726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef15fa-7558-414a-903b-2a30bf363bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceba0fd5aef942ca97a15cf52258ae16, user_id=78cc9bddddbd4111ba0cfdf0cd9ed12f, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:28.883420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef15fa-7558-414a-903b-2a30bf363bda admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:28.974696 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1011/2046] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:28 2022] PUT /identity/v3/projects/ceba0fd5aef942ca97a15cf52258ae16/users/78cc9bddddbd4111ba0cfdf0cd9ed12f/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:28.975556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef15fa-7558-414a-903b-2a30bf363bda admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:28.976195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1036/2047] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:28 2022] POST /identity/v3/projects => generated 387 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:28.977374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b53265-e9d0-4345-bb44-947da71fbf86 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.024678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:29.045015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef15fa-7558-414a-903b-2a30bf363bda admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.047859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b53265-e9d0-4345-bb44-947da71fbf86 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.050761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.052184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.052637 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.053539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] PATH_INFO: `/v3/users/ebb96d98580040ac88d469bbb2f313b5` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.055526 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd121ab-a78e-4331-b900-6ff95e3de378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.056219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd121ab-a78e-4331-b900-6ff95e3de378 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.056376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:29.056512 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd121ab-a78e-4331-b900-6ff95e3de378 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.056829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd121ab-a78e-4331-b900-6ff95e3de378 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.057465 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd121ab-a78e-4331-b900-6ff95e3de378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.058006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebb96d98580040ac88d469bbb2f313b5)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.059971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd121ab-a78e-4331-b900-6ff95e3de378 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.060199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.128275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1037/2048] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:28 2022] POST /identity/v3/users => generated 306 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:29.129795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd121ab-a78e-4331-b900-6ff95e3de378 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.139603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd121ab-a78e-4331-b900-6ff95e3de378 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.144403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd348ee-982b-4c84-ae3f-6886b1ff32a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.145573 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd348ee-982b-4c84-ae3f-6886b1ff32a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.146096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd348ee-982b-4c84-ae3f-6886b1ff32a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.148328 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:29.151731 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd348ee-982b-4c84-ae3f-6886b1ff32a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.151731 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd348ee-982b-4c84-ae3f-6886b1ff32a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.151731 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd348ee-982b-4c84-ae3f-6886b1ff32a2 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.161793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1038/2049] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:29.162669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd348ee-982b-4c84-ae3f-6886b1ff32a2 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.172859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd348ee-982b-4c84-ae3f-6886b1ff32a2 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.178061 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453adb1-f5b6-46e1-868c-c17cb921aa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.178704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453adb1-f5b6-46e1-868c-c17cb921aa4d admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.178988 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453adb1-f5b6-46e1-868c-c17cb921aa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.179279 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453adb1-f5b6-46e1-868c-c17cb921aa4d admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.179918 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453adb1-f5b6-46e1-868c-c17cb921aa4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.181819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453adb1-f5b6-46e1-868c-c17cb921aa4d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.227958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebb96d98580040ac88d469bbb2f313b5'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:29.323368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1039/2050] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:29 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:29.324205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453adb1-f5b6-46e1-868c-c17cb921aa4d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.334237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebb96d98580040ac88d469bbb2f313b5'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:29.335141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebb96d98580040ac88d469bbb2f313b5'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:29.335382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453adb1-f5b6-46e1-868c-c17cb921aa4d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.335808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebb96d98580040ac88d469bbb2f313b5'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:29.340016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49345572-bfca-494a-a25d-36beb18dc939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.343756 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49345572-bfca-494a-a25d-36beb18dc939 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.343756 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49345572-bfca-494a-a25d-36beb18dc939 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.343756 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49345572-bfca-494a-a25d-36beb18dc939 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.343756 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49345572-bfca-494a-a25d-36beb18dc939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.343756 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49345572-bfca-494a-a25d-36beb18dc939 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.344217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebb96d98580040ac88d469bbb2f313b5'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:29.353801 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1040/2051] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:29.354649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49345572-bfca-494a-a25d-36beb18dc939 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.357112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-49345572-bfca-494a-a25d-36beb18dc939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:29.366438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1012/2052] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:28 2022] DELETE /identity/v3/users/ebb96d98580040ac88d469bbb2f313b5 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:29.366438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.366438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:29.370670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49345572-bfca-494a-a25d-36beb18dc939 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.430729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6726cd1-e56e-452f-9eda-04a118044001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.431308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6726cd1-e56e-452f-9eda-04a118044001 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.431481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c31746-6fe8-4515-9ce0-58dbe0b80289 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.431581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6726cd1-e56e-452f-9eda-04a118044001 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.431843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6726cd1-e56e-452f-9eda-04a118044001 admin admin] PATH_INFO: `/v3/projects/ceba0fd5aef942ca97a15cf52258ae16/users/78cc9bddddbd4111ba0cfdf0cd9ed12f/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.433786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6726cd1-e56e-452f-9eda-04a118044001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceba0fd5aef942ca97a15cf52258ae16, user_id=78cc9bddddbd4111ba0cfdf0cd9ed12f, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.436188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb0c31-673d-498c-8f03-d3286f577c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.436796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb0c31-673d-498c-8f03-d3286f577c5b admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.441906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb0c31-673d-498c-8f03-d3286f577c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.441906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb0c31-673d-498c-8f03-d3286f577c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.442237 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6726cd1-e56e-452f-9eda-04a118044001 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.442341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb0c31-673d-498c-8f03-d3286f577c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.442341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb0c31-673d-498c-8f03-d3286f577c5b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.449173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1041/2053] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:29 2022] PUT /identity/v3/projects/ceba0fd5aef942ca97a15cf52258ae16/users/78cc9bddddbd4111ba0cfdf0cd9ed12f/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:29.450042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6726cd1-e56e-452f-9eda-04a118044001 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.451130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1013/2054] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:29.452320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f6726cd1-e56e-452f-9eda-04a118044001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:29.455576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eb089-5df1-403c-b9a0-bbccf7e478a2 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.455870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eb089-5df1-403c-b9a0-bbccf7e478a2 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.456145 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eb089-5df1-403c-b9a0-bbccf7e478a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.465547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:29.477535 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-340eb089-5df1-403c-b9a0-bbccf7e478a2 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:29.478027 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:29.478748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-340eb089-5df1-403c-b9a0-bbccf7e478a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:29.524125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6726cd1-e56e-452f-9eda-04a118044001 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.529173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb212f43-3a7e-43c8-af9a-533da650daa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.529834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb212f43-3a7e-43c8-af9a-533da650daa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.530110 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb212f43-3a7e-43c8-af9a-533da650daa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.530415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb212f43-3a7e-43c8-af9a-533da650daa7 admin admin] PATH_INFO: `/v3/projects/2ce3a06716b044549ee8f11b99430c72/users/16eb1dbb721f43e7882219a8ae5dbb03/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.538474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1014/2055] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:29.539497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eb089-5df1-403c-b9a0-bbccf7e478a2 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.545556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb212f43-3a7e-43c8-af9a-533da650daa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce3a06716b044549ee8f11b99430c72, user_id=16eb1dbb721f43e7882219a8ae5dbb03, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.548713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb212f43-3a7e-43c8-af9a-533da650daa7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.549214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eb089-5df1-403c-b9a0-bbccf7e478a2 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.554878 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a70e8-826f-4dd2-a5cb-cb2387b7d7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.555622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a70e8-826f-4dd2-a5cb-cb2387b7d7f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.555904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a70e8-826f-4dd2-a5cb-cb2387b7d7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.556220 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a70e8-826f-4dd2-a5cb-cb2387b7d7f2 admin admin] PATH_INFO: `/v3/projects/95ca05c04b284cd5bb894aba90f17f20/users/77cf86c49550462e887ddcc7e73982fe/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.573332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a70e8-826f-4dd2-a5cb-cb2387b7d7f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ca05c04b284cd5bb894aba90f17f20, user_id=77cf86c49550462e887ddcc7e73982fe, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.575988 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a70e8-826f-4dd2-a5cb-cb2387b7d7f2 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.602397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1042/2056] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:29 2022] PUT /identity/v3/projects/2ce3a06716b044549ee8f11b99430c72/users/16eb1dbb721f43e7882219a8ae5dbb03/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:29.603770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c36f4-8ccd-4d1d-b9a3-5d7d11f6ad2f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.604052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c36f4-8ccd-4d1d-b9a3-5d7d11f6ad2f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.604317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c36f4-8ccd-4d1d-b9a3-5d7d11f6ad2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.631620 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-231c36f4-8ccd-4d1d-b9a3-5d7d11f6ad2f None None] MFA Rules not processed for user `78cc9bddddbd4111ba0cfdf0cd9ed12f`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:29.633021 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-231c36f4-8ccd-4d1d-b9a3-5d7d11f6ad2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:29.636546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1015/2057] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:29 2022] PUT /identity/v3/projects/95ca05c04b284cd5bb894aba90f17f20/users/77cf86c49550462e887ddcc7e73982fe/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:29.637704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a70e8-826f-4dd2-a5cb-cb2387b7d7f2 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.639857 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-257a70e8-826f-4dd2-a5cb-cb2387b7d7f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:29.654359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:29.695868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:29.701127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a70e8-826f-4dd2-a5cb-cb2387b7d7f2 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.705906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b261e3-f47b-4bb8-9cf3-6151866b5dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.712712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b261e3-f47b-4bb8-9cf3-6151866b5dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.712886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1043/2058] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:29.712985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b261e3-f47b-4bb8-9cf3-6151866b5dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.712985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b261e3-f47b-4bb8-9cf3-6151866b5dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.713165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c36f4-8ccd-4d1d-b9a3-5d7d11f6ad2f None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.780138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c36f4-8ccd-4d1d-b9a3-5d7d11f6ad2f None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.781479 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b261e3-f47b-4bb8-9cf3-6151866b5dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.782971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b261e3-f47b-4bb8-9cf3-6151866b5dc7 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.784787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe2d18-b967-4dfd-b693-dc74c13956f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.786383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe2d18-b967-4dfd-b693-dc74c13956f0 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.786734 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe2d18-b967-4dfd-b693-dc74c13956f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.787016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe2d18-b967-4dfd-b693-dc74c13956f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.787671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe2d18-b967-4dfd-b693-dc74c13956f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.789242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe2d18-b967-4dfd-b693-dc74c13956f0 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.793466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1016/2059] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:29.794548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b261e3-f47b-4bb8-9cf3-6151866b5dc7 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.802654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1044/2060] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:29.802654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe2d18-b967-4dfd-b693-dc74c13956f0 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.803043 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d5fe2d18-b967-4dfd-b693-dc74c13956f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:29.809972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b261e3-f47b-4bb8-9cf3-6151866b5dc7 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.815386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba7ebd-cfab-43f7-ba0c-6b96d3dbd2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.816515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba7ebd-cfab-43f7-ba0c-6b96d3dbd2e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.816515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba7ebd-cfab-43f7-ba0c-6b96d3dbd2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.817412 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba7ebd-cfab-43f7-ba0c-6b96d3dbd2e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.818275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba7ebd-cfab-43f7-ba0c-6b96d3dbd2e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.819902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba7ebd-cfab-43f7-ba0c-6b96d3dbd2e1 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.833099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1017/2061] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:29.833918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba7ebd-cfab-43f7-ba0c-6b96d3dbd2e1 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.845091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba7ebd-cfab-43f7-ba0c-6b96d3dbd2e1 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.850065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1539ab-149a-4c56-92e2-8a0723cc14d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.850745 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1539ab-149a-4c56-92e2-8a0723cc14d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.851028 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1539ab-149a-4c56-92e2-8a0723cc14d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.851303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1539ab-149a-4c56-92e2-8a0723cc14d9 admin admin] PATH_INFO: `/v3/projects/2ce3a06716b044549ee8f11b99430c72/users/16eb1dbb721f43e7882219a8ae5dbb03/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.853250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1539ab-149a-4c56-92e2-8a0723cc14d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce3a06716b044549ee8f11b99430c72, user_id=16eb1dbb721f43e7882219a8ae5dbb03, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.856019 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1539ab-149a-4c56-92e2-8a0723cc14d9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.867861 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1018/2062] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:29 2022] PUT /identity/v3/projects/2ce3a06716b044549ee8f11b99430c72/users/16eb1dbb721f43e7882219a8ae5dbb03/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:29.870391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:29.871445 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe2d18-b967-4dfd-b693-dc74c13956f0 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.875486 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1539ab-149a-4c56-92e2-8a0723cc14d9 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:29.929373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24e979-3baa-4bf2-8522-b3956f17e414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.930193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24e979-3baa-4bf2-8522-b3956f17e414 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.930475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24e979-3baa-4bf2-8522-b3956f17e414 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.930775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24e979-3baa-4bf2-8522-b3956f17e414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.941631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1539ab-149a-4c56-92e2-8a0723cc14d9 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:29.946682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3796c-db24-4099-987f-4adacc10adb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:29.947356 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3796c-db24-4099-987f-4adacc10adb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.947643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3796c-db24-4099-987f-4adacc10adb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.947923 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3796c-db24-4099-987f-4adacc10adb9 admin admin] PATH_INFO: `/v3/projects/95ca05c04b284cd5bb894aba90f17f20/users/77cf86c49550462e887ddcc7e73982fe/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.949911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3796c-db24-4099-987f-4adacc10adb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ca05c04b284cd5bb894aba90f17f20, user_id=77cf86c49550462e887ddcc7e73982fe, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.952936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3796c-db24-4099-987f-4adacc10adb9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:29.958320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:29.964053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1019/2063] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:29 2022] PUT /identity/v3/projects/95ca05c04b284cd5bb894aba90f17f20/users/77cf86c49550462e887ddcc7e73982fe/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:29.965323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614eb2-8e34-4164-8b29-4271a048a081 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:29.965606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614eb2-8e34-4164-8b29-4271a048a081 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:29.965878 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614eb2-8e34-4164-8b29-4271a048a081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:29.996426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24e979-3baa-4bf2-8522-b3956f17e414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:29.998268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24e979-3baa-4bf2-8522-b3956f17e414 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:30.001790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-d5614eb2-8e34-4164-8b29-4271a048a081 None None] MFA Rules not processed for user `16eb1dbb721f43e7882219a8ae5dbb03`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:30.003147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d5614eb2-8e34-4164-8b29-4271a048a081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:30.011258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1045/2064] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:30.018145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a35cc6-3593-4625-839c-2f661e78aa10 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:30.018145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a35cc6-3593-4625-839c-2f661e78aa10 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:30.018145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a35cc6-3593-4625-839c-2f661e78aa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:30.059164 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-35a35cc6-3593-4625-839c-2f661e78aa10 None None] MFA Rules not processed for user `77cf86c49550462e887ddcc7e73982fe`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:30.059164 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-35a35cc6-3593-4625-839c-2f661e78aa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:30.124253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:30.134835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1020/2065] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:29 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:30.143015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1046/2066] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:30.147932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5614eb2-8e34-4164-8b29-4271a048a081 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:30.156850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:30.164525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a35cc6-3593-4625-839c-2f661e78aa10 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:30.166003 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-35a35cc6-3593-4625-839c-2f661e78aa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:30.212371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5614eb2-8e34-4164-8b29-4271a048a081 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:30.216139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd09c1e-427e-4f6e-8ff8-8febdf06e7fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:30.216751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd09c1e-427e-4f6e-8ff8-8febdf06e7fc service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:30.217033 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd09c1e-427e-4f6e-8ff8-8febdf06e7fc service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:30.217314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd09c1e-427e-4f6e-8ff8-8febdf06e7fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:30.226337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd09c1e-427e-4f6e-8ff8-8febdf06e7fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:30.227311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a35cc6-3593-4625-839c-2f661e78aa10 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:30.231289 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd09c1e-427e-4f6e-8ff8-8febdf06e7fc service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:30.231435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61c7c3-74ca-482b-a73a-f07962c48c1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:30.231737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61c7c3-74ca-482b-a73a-f07962c48c1b service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:30.232011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61c7c3-74ca-482b-a73a-f07962c48c1b service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:30.232278 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61c7c3-74ca-482b-a73a-f07962c48c1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:30.238192 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1021/2067] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:30 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:30.245044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61c7c3-74ca-482b-a73a-f07962c48c1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:30.246827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61c7c3-74ca-482b-a73a-f07962c48c1b service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:30.258164 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1047/2068] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:57:30.290973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd09c1e-427e-4f6e-8ff8-8febdf06e7fc service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:30.292602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61c7c3-74ca-482b-a73a-f07962c48c1b service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:30.294944 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-fcd09c1e-427e-4f6e-8ff8-8febdf06e7fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:30.412584 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd09c1e-427e-4f6e-8ff8-8febdf06e7fc service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:30.416505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61c7c3-74ca-482b-a73a-f07962c48c1b service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:30.421604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:30.422253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:30.422471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:30.422871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:30.422969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:30.423367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] PATH_INFO: `/v3/projects/d0bcf4149ec14ca3abbde516fe1b807a` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:30.423489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:30.423817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] PATH_INFO: `/v3/users/5faa1995e8d34499bc63b2564e30758c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:30.424712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0bcf4149ec14ca3abbde516fe1b807a)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:30.425170 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5faa1995e8d34499bc63b2564e30758c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:30.427415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:30.428176 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:30.560886 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0bcf4149ec14ca3abbde516fe1b807a', 'request_id': 'req-1b7eaa26-1dab-4b7f-8856-f38a7341f557'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:30.568031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0bcf4149ec14ca3abbde516fe1b807a', 'request_id': 'req-1b7eaa26-1dab-4b7f-8856-f38a7341f557'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:30.571108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:30.647203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:30.709735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0bcf4149ec14ca3abbde516fe1b807a', 'request_id': 'req-1b7eaa26-1dab-4b7f-8856-f38a7341f557'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:30.724340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] The token cache is being invalidate because project d0bcf4149ec14ca3abbde516fe1b807a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:30.724553 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:30.724654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5faa1995e8d34499bc63b2564e30758c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:30.725329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:30.727180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1022/2069] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:30 2022] DELETE /identity/v3/projects/d0bcf4149ec14ca3abbde516fe1b807a => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:30.730499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:30.732025 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:30.783105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5faa1995e8d34499bc63b2564e30758c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:30.783730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5faa1995e8d34499bc63b2564e30758c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:30.784342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5faa1995e8d34499bc63b2564e30758c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:30.796970 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5faa1995e8d34499bc63b2564e30758c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:30.804100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7eaa26-1dab-4b7f-8856-f38a7341f557 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:30.809296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633f7e3-b8fb-48d8-be66-da2b3add0aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:30.809938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633f7e3-b8fb-48d8-be66-da2b3add0aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:30.810220 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633f7e3-b8fb-48d8-be66-da2b3add0aae admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:30.810520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633f7e3-b8fb-48d8-be66-da2b3add0aae admin admin] PATH_INFO: `/v3/users/b459c5ca6d524f5a9de282f8fc975a1a` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:30.811121 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1048/2070] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:30 2022] DELETE /identity/v3/users/5faa1995e8d34499bc63b2564e30758c => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:30.812031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:30.814694 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633f7e3-b8fb-48d8-be66-da2b3add0aae admin admin] RBAC: Authorizing `identity:delete_user(user_id=b459c5ca6d524f5a9de282f8fc975a1a)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:30.814986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:30.818147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633f7e3-b8fb-48d8-be66-da2b3add0aae admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:30.884874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dac42d-9c63-4f1e-a96d-b778040a197c admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:30.889052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8f271-fb98-476f-898f-deae28f2100e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:30.889703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8f271-fb98-476f-898f-deae28f2100e service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:30.889983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8f271-fb98-476f-898f-deae28f2100e service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:30.890257 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8f271-fb98-476f-898f-deae28f2100e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:30.892046 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-aeb8f271-fb98-476f-898f-deae28f2100e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:30.953011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8f271-fb98-476f-898f-deae28f2100e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:30.954692 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8f271-fb98-476f-898f-deae28f2100e service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:30.965209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1049/2071] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:30 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:30.966200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8f271-fb98-476f-898f-deae28f2100e service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:30.967684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-aeb8f271-fb98-476f-898f-deae28f2100e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:30.998732 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:31.012788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0633f7e3-b8fb-48d8-be66-da2b3add0aae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b459c5ca6d524f5a9de282f8fc975a1a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:31.044544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8f271-fb98-476f-898f-deae28f2100e service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:31.049132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:31.049748 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:31.050024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:31.050295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] PATH_INFO: `/v3/users/7bba8cd408aa4a219fe35c8380ba17ed` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:31.052109 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bba8cd408aa4a219fe35c8380ba17ed)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:31.054504 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:31.057206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:31.087141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0633f7e3-b8fb-48d8-be66-da2b3add0aae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b459c5ca6d524f5a9de282f8fc975a1a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:31.088164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0633f7e3-b8fb-48d8-be66-da2b3add0aae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b459c5ca6d524f5a9de282f8fc975a1a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:31.088866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0633f7e3-b8fb-48d8-be66-da2b3add0aae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b459c5ca6d524f5a9de282f8fc975a1a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:31.096371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0633f7e3-b8fb-48d8-be66-da2b3add0aae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b459c5ca6d524f5a9de282f8fc975a1a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:31.115432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1023/2072] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:30 2022] DELETE /identity/v3/users/b459c5ca6d524f5a9de282f8fc975a1a => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:31.116832 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ac33e-11ee-4cd0-88c0-68267922cf77 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:31.117181 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ac33e-11ee-4cd0-88c0-68267922cf77 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:31.117548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ac33e-11ee-4cd0-88c0-68267922cf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:31.138956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-6c8ac33e-11ee-4cd0-88c0-68267922cf77 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:31.140162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6c8ac33e-11ee-4cd0-88c0-68267922cf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:31.199578 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1024/2073] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:31.200904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91099a-3f1d-4ee3-87c7-d2b7140bfa08 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:31.201130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91099a-3f1d-4ee3-87c7-d2b7140bfa08 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:31.201406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91099a-3f1d-4ee3-87c7-d2b7140bfa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:31.221290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ae91099a-3f1d-4ee3-87c7-d2b7140bfa08 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:31.222506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ae91099a-3f1d-4ee3-87c7-d2b7140bfa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:31.234046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1025/2074] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:31.234046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91099a-3f1d-4ee3-87c7-d2b7140bfa08 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:31.235254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ae91099a-3f1d-4ee3-87c7-d2b7140bfa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:31.280048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bba8cd408aa4a219fe35c8380ba17ed'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:31.303401 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91099a-3f1d-4ee3-87c7-d2b7140bfa08 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:31.317547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955e724-95b3-4412-a7bf-41282be1d8d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:31.318203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e724-95b3-4412-a7bf-41282be1d8d7 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:31.318504 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e724-95b3-4412-a7bf-41282be1d8d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:31.318801 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e724-95b3-4412-a7bf-41282be1d8d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:31.329972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955e724-95b3-4412-a7bf-41282be1d8d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:31.331541 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955e724-95b3-4412-a7bf-41282be1d8d7 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:31.341803 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1026/2075] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:31.342651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955e724-95b3-4412-a7bf-41282be1d8d7 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:31.351190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bba8cd408aa4a219fe35c8380ba17ed'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:31.351854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955e724-95b3-4412-a7bf-41282be1d8d7 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:31.352337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bba8cd408aa4a219fe35c8380ba17ed'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:31.353212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bba8cd408aa4a219fe35c8380ba17ed'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:31.355637 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6275e-4279-46e5-bd82-bd7f69d049bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:31.361091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6275e-4279-46e5-bd82-bd7f69d049bb service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:31.361091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6275e-4279-46e5-bd82-bd7f69d049bb service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:31.361091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6275e-4279-46e5-bd82-bd7f69d049bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:31.361091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-88a6275e-4279-46e5-bd82-bd7f69d049bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:31.370518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bba8cd408aa4a219fe35c8380ba17ed'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:31.389902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:31.398209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6275e-4279-46e5-bd82-bd7f69d049bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:31.399734 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6275e-4279-46e5-bd82-bd7f69d049bb service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:31.403587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1050/2076] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:30 2022] DELETE /identity/v3/users/7bba8cd408aa4a219fe35c8380ba17ed => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:31.437571 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1027/2077] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:31.660162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:32.067366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:32.068871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:32.171103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2df2b8-3d2d-4bb4-b895-1b1bfb47c2cc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:32.175808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:32.178525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:32.178525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:32.178525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] PATH_INFO: `/v3/projects/aea3816cd39d4bcda0818af6e505a207` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:32.179491 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea3816cd39d4bcda0818af6e505a207)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:32.181310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6275e-4279-46e5-bd82-bd7f69d049bb service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:32.181769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:32.183438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-88a6275e-4279-46e5-bd82-bd7f69d049bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:32.197509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6275e-4279-46e5-bd82-bd7f69d049bb service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:32.202421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:32.203361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:32.203641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:32.203912 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] PATH_INFO: `/v3/projects/2630da85cb8d4f0a9d01897f20152567` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:32.205045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2630da85cb8d4f0a9d01897f20152567)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:32.207150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:32.330034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea3816cd39d4bcda0818af6e505a207', 'request_id': 'req-d2e520de-d184-4398-b110-9fa1dc31cca3'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.349614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea3816cd39d4bcda0818af6e505a207', 'request_id': 'req-d2e520de-d184-4398-b110-9fa1dc31cca3'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.365339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2630da85cb8d4f0a9d01897f20152567', 'request_id': 'req-6c52fcee-d977-4f24-a017-6c760d2e9ee2'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.382465 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2630da85cb8d4f0a9d01897f20152567', 'request_id': 'req-6c52fcee-d977-4f24-a017-6c760d2e9ee2'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.401867 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:32.431509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea3816cd39d4bcda0818af6e505a207', 'request_id': 'req-d2e520de-d184-4398-b110-9fa1dc31cca3'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.450080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] The token cache is being invalidate because project aea3816cd39d4bcda0818af6e505a207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:32.450369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.451071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.453245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1051/2078] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:32 2022] DELETE /identity/v3/projects/aea3816cd39d4bcda0818af6e505a207 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:32.453778 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:32.455116 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:32.505133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2630da85cb8d4f0a9d01897f20152567', 'request_id': 'req-6c52fcee-d977-4f24-a017-6c760d2e9ee2'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.520810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] The token cache is being invalidate because project 2630da85cb8d4f0a9d01897f20152567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:32.521293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.522326 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.523965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1028/2079] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:32 2022] DELETE /identity/v3/projects/2630da85cb8d4f0a9d01897f20152567 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:32.525227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:32.527075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:32.529781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e520de-d184-4398-b110-9fa1dc31cca3 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:32.590974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:32.596592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:32.596592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:32.596592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] PATH_INFO: `/v3/projects/9f48d391ca7c438b94c9ba699c0eb724` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:32.596592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f48d391ca7c438b94c9ba699c0eb724)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:32.596918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52fcee-d977-4f24-a017-6c760d2e9ee2 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:32.597039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:32.599585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac63aa6-5f5b-4228-b9fa-42ff78c12374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:32.600218 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac63aa6-5f5b-4228-b9fa-42ff78c12374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:32.600496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac63aa6-5f5b-4228-b9fa-42ff78c12374 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:32.600765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac63aa6-5f5b-4228-b9fa-42ff78c12374 admin admin] PATH_INFO: `/v3/users/77cf86c49550462e887ddcc7e73982fe` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:32.601972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac63aa6-5f5b-4228-b9fa-42ff78c12374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77cf86c49550462e887ddcc7e73982fe)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:32.603893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac63aa6-5f5b-4228-b9fa-42ff78c12374 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:32.677822 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f48d391ca7c438b94c9ba699c0eb724', 'request_id': 'req-d9943ca3-cfee-4694-bc55-5e44425bed45'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.684398 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f48d391ca7c438b94c9ba699c0eb724', 'request_id': 'req-d9943ca3-cfee-4694-bc55-5e44425bed45'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.699222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:32.758643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5ac63aa6-5f5b-4228-b9fa-42ff78c12374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77cf86c49550462e887ddcc7e73982fe'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.774367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f48d391ca7c438b94c9ba699c0eb724', 'request_id': 'req-d9943ca3-cfee-4694-bc55-5e44425bed45'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.787646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] The token cache is being invalidate because project 9f48d391ca7c438b94c9ba699c0eb724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:32.787963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.788658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.790346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1052/2080] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:32 2022] DELETE /identity/v3/projects/9f48d391ca7c438b94c9ba699c0eb724 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:32.795718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:32.795718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:32.867976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9943ca3-cfee-4694-bc55-5e44425bed45 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:32.872809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:32.873434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:32.873711 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:32.873978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] PATH_INFO: `/v3/users/78cc9bddddbd4111ba0cfdf0cd9ed12f` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:32.875163 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78cc9bddddbd4111ba0cfdf0cd9ed12f)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:32.877069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:32.881072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5ac63aa6-5f5b-4228-b9fa-42ff78c12374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77cf86c49550462e887ddcc7e73982fe'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.886072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5ac63aa6-5f5b-4228-b9fa-42ff78c12374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77cf86c49550462e887ddcc7e73982fe'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.886072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5ac63aa6-5f5b-4228-b9fa-42ff78c12374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77cf86c49550462e887ddcc7e73982fe'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.888033 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:32.896142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5ac63aa6-5f5b-4228-b9fa-42ff78c12374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77cf86c49550462e887ddcc7e73982fe'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:32.931438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1029/2081] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:32 2022] DELETE /identity/v3/users/77cf86c49550462e887ddcc7e73982fe => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:32.937736 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ec1a9-53b7-4428-aa64-f9877c52dfaf None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:32.937736 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ec1a9-53b7-4428-aa64-f9877c52dfaf None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:32.937736 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ec1a9-53b7-4428-aa64-f9877c52dfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:32.960431 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-159ec1a9-53b7-4428-aa64-f9877c52dfaf None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:32.961849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-159ec1a9-53b7-4428-aa64-f9877c52dfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:33.060607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1030/2082] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:33.067961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fab5b2-bb3f-4f3e-a3fd-6f2947de713b None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:33.068250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fab5b2-bb3f-4f3e-a3fd-6f2947de713b None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:33.068530 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fab5b2-bb3f-4f3e-a3fd-6f2947de713b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:33.091739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-49fab5b2-bb3f-4f3e-a3fd-6f2947de713b None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:33.092926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-49fab5b2-bb3f-4f3e-a3fd-6f2947de713b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:33.103182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1031/2083] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:33.105606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fab5b2-bb3f-4f3e-a3fd-6f2947de713b None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:33.116571 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fab5b2-bb3f-4f3e-a3fd-6f2947de713b None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:33.122465 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc883c0-897f-4111-a229-709c936a8261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:33.123219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc883c0-897f-4111-a229-709c936a8261 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:33.123501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc883c0-897f-4111-a229-709c936a8261 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:33.123771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc883c0-897f-4111-a229-709c936a8261 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:33.124505 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc883c0-897f-4111-a229-709c936a8261 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:33.124813 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc883c0-897f-4111-a229-709c936a8261 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:33.126518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc883c0-897f-4111-a229-709c936a8261 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:33.146674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1032/2084] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:57:33.154402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc883c0-897f-4111-a229-709c936a8261 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:33.157649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78cc9bddddbd4111ba0cfdf0cd9ed12f'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.167835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc883c0-897f-4111-a229-709c936a8261 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:33.179662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d4466-8002-4db1-be0a-625c77e3e10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:33.180631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d4466-8002-4db1-be0a-625c77e3e10d admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:33.181090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d4466-8002-4db1-be0a-625c77e3e10d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:33.181534 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d4466-8002-4db1-be0a-625c77e3e10d admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:33.183188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d4466-8002-4db1-be0a-625c77e3e10d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:33.185544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d4466-8002-4db1-be0a-625c77e3e10d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:33.246078 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78cc9bddddbd4111ba0cfdf0cd9ed12f'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.246725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78cc9bddddbd4111ba0cfdf0cd9ed12f'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.247339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78cc9bddddbd4111ba0cfdf0cd9ed12f'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.260705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78cc9bddddbd4111ba0cfdf0cd9ed12f'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.273461 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1053/2085] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:32 2022] DELETE /identity/v3/users/78cc9bddddbd4111ba0cfdf0cd9ed12f => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:33.292998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1033/2086] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:33 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:57:33.294191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:33.295638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:33.367371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd44895-5871-4578-bf8c-d24fb5496559 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:33.429371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff0098-c5a8-4154-b97b-82f78d59b82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:33.429997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff0098-c5a8-4154-b97b-82f78d59b82c admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:33.430280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff0098-c5a8-4154-b97b-82f78d59b82c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:33.430587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff0098-c5a8-4154-b97b-82f78d59b82c admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:33.431227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff0098-c5a8-4154-b97b-82f78d59b82c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:33.433272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff0098-c5a8-4154-b97b-82f78d59b82c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:33.444328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d4466-8002-4db1-be0a-625c77e3e10d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:33.446152 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-243d4466-8002-4db1-be0a-625c77e3e10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:33.459881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d4466-8002-4db1-be0a-625c77e3e10d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:33.473385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:33.473385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:33.473385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:33.473385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] PATH_INFO: `/v3/users/b136a9a7f0324a34b6aa28ca59db6da3` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:33.474901 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b136a9a7f0324a34b6aa28ca59db6da3)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:33.477184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:33.552547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1054/2087] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:33 2022] POST /identity/v3/users => generated 286 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:33.553653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff0098-c5a8-4154-b97b-82f78d59b82c admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:33.556076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-bcff0098-c5a8-4154-b97b-82f78d59b82c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:33.573532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff0098-c5a8-4154-b97b-82f78d59b82c admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:33.580938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:33.581593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:33.581927 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:33.582249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] PATH_INFO: `/v3/projects/95ca05c04b284cd5bb894aba90f17f20` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:33.583450 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] RBAC: Authorizing `identity:delete_project(project_id=95ca05c04b284cd5bb894aba90f17f20)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:33.585899 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:33.713233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b136a9a7f0324a34b6aa28ca59db6da3'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.757272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95ca05c04b284cd5bb894aba90f17f20', 'request_id': 'req-ab7566fa-4331-46d5-9cb5-c80db1a047df'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.764372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95ca05c04b284cd5bb894aba90f17f20', 'request_id': 'req-ab7566fa-4331-46d5-9cb5-c80db1a047df'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.767597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b136a9a7f0324a34b6aa28ca59db6da3'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.768430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b136a9a7f0324a34b6aa28ca59db6da3'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.772819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b136a9a7f0324a34b6aa28ca59db6da3'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.782399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b136a9a7f0324a34b6aa28ca59db6da3'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.809313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1034/2088] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:33 2022] DELETE /identity/v3/users/b136a9a7f0324a34b6aa28ca59db6da3 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:33.810372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:33.811715 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:33.829985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95ca05c04b284cd5bb894aba90f17f20', 'request_id': 'req-ab7566fa-4331-46d5-9cb5-c80db1a047df'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.842548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] The token cache is being invalidate because project 95ca05c04b284cd5bb894aba90f17f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:33.842847 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.843580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:33.845282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1055/2089] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:33 2022] DELETE /identity/v3/projects/95ca05c04b284cd5bb894aba90f17f20 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:33.849897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:33.853015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:33.888328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cd34f-5f02-4247-8446-9f101409254a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:33.893666 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e832c-4a0b-4301-8fd4-a36fe2bdcb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:33.894270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e832c-4a0b-4301-8fd4-a36fe2bdcb8b admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:33.895123 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e832c-4a0b-4301-8fd4-a36fe2bdcb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:33.895399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e832c-4a0b-4301-8fd4-a36fe2bdcb8b admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:33.896038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e832c-4a0b-4301-8fd4-a36fe2bdcb8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:33.897626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e832c-4a0b-4301-8fd4-a36fe2bdcb8b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:33.907655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1035/2090] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:33.908826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f14a-8982-463c-a0f5-c89cf0252daf None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:33.909111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f14a-8982-463c-a0f5-c89cf0252daf None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:33.909395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f14a-8982-463c-a0f5-c89cf0252daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:33.923735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7566fa-4331-46d5-9cb5-c80db1a047df admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:33.928651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-0496f14a-8982-463c-a0f5-c89cf0252daf None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:33.929230 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39638fe5-9510-405b-a92c-cbaaee3c03d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:33.930425 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0496f14a-8982-463c-a0f5-c89cf0252daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:33.930538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39638fe5-9510-405b-a92c-cbaaee3c03d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:33.930538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39638fe5-9510-405b-a92c-cbaaee3c03d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:33.930538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39638fe5-9510-405b-a92c-cbaaee3c03d7 admin admin] PATH_INFO: `/v3/projects/ceba0fd5aef942ca97a15cf52258ae16` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:33.931594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39638fe5-9510-405b-a92c-cbaaee3c03d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceba0fd5aef942ca97a15cf52258ae16)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:33.933937 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39638fe5-9510-405b-a92c-cbaaee3c03d7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:33.938463 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1036/2091] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:33.939649 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eef14c-b946-4843-bcfe-6eed670e06fe None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:33.939932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eef14c-b946-4843-bcfe-6eed670e06fe None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:33.940582 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eef14c-b946-4843-bcfe-6eed670e06fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:33.963072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-18eef14c-b946-4843-bcfe-6eed670e06fe None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:33.963793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-18eef14c-b946-4843-bcfe-6eed670e06fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:33.978180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1037/2092] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:33.979061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eef14c-b946-4843-bcfe-6eed670e06fe None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:33.980451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-18eef14c-b946-4843-bcfe-6eed670e06fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:33.993141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eef14c-b946-4843-bcfe-6eed670e06fe None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:33.999521 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc41e75-c18d-4335-bd21-05dee505fc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:34.000130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc41e75-c18d-4335-bd21-05dee505fc14 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.000411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc41e75-c18d-4335-bd21-05dee505fc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.000682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc41e75-c18d-4335-bd21-05dee505fc14 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.001345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc41e75-c18d-4335-bd21-05dee505fc14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:34.002970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc41e75-c18d-4335-bd21-05dee505fc14 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:34.013883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1038/2093] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:34.014735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc41e75-c18d-4335-bd21-05dee505fc14 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:34.017932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-8dc41e75-c18d-4335-bd21-05dee505fc14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:34.084110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc41e75-c18d-4335-bd21-05dee505fc14 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:34.085993 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-39638fe5-9510-405b-a92c-cbaaee3c03d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceba0fd5aef942ca97a15cf52258ae16', 'request_id': 'req-39638fe5-9510-405b-a92c-cbaaee3c03d7'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:34.092316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-39638fe5-9510-405b-a92c-cbaaee3c03d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceba0fd5aef942ca97a15cf52258ae16', 'request_id': 'req-39638fe5-9510-405b-a92c-cbaaee3c03d7'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:34.133990 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389c27c-edcb-4424-a58b-7c64d2b47870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:34.134731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389c27c-edcb-4424-a58b-7c64d2b47870 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.135011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389c27c-edcb-4424-a58b-7c64d2b47870 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.135283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389c27c-edcb-4424-a58b-7c64d2b47870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.149256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:34.177874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-39638fe5-9510-405b-a92c-cbaaee3c03d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceba0fd5aef942ca97a15cf52258ae16', 'request_id': 'req-39638fe5-9510-405b-a92c-cbaaee3c03d7'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:34.188860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-39638fe5-9510-405b-a92c-cbaaee3c03d7 admin admin] The token cache is being invalidate because project ceba0fd5aef942ca97a15cf52258ae16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:34.189150 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-39638fe5-9510-405b-a92c-cbaaee3c03d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:34.195098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-39638fe5-9510-405b-a92c-cbaaee3c03d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:34.195098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1056/2094] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:33 2022] DELETE /identity/v3/projects/ceba0fd5aef942ca97a15cf52258ae16 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:34.195098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8f69a-0551-4b0b-b757-048b420e859d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.195098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8f69a-0551-4b0b-b757-048b420e859d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.195098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8f69a-0551-4b0b-b757-048b420e859d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.199231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389c27c-edcb-4424-a58b-7c64d2b47870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:34.200661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389c27c-edcb-4424-a58b-7c64d2b47870 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:34.211170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1039/2095] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:34.211962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389c27c-edcb-4424-a58b-7c64d2b47870 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:34.221025 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389c27c-edcb-4424-a58b-7c64d2b47870 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:34.221517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-76b8f69a-0551-4b0b-b757-048b420e859d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:34.223380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-76b8f69a-0551-4b0b-b757-048b420e859d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:34.226648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf902804-d824-4c13-985d-c0b63c2b7430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:34.227382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf902804-d824-4c13-985d-c0b63c2b7430 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.227562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf902804-d824-4c13-985d-c0b63c2b7430 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.227835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf902804-d824-4c13-985d-c0b63c2b7430 admin admin] PATH_INFO: `/v3/projects/088ddf9a768249a5b16e55c6e01c6e39/users/20b9b75717a745a18973bd43468d579a/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.233301 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1057/2096] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:34.237584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281565b0-1517-437c-9cf1-7768831a0eb8 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.237911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281565b0-1517-437c-9cf1-7768831a0eb8 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.238553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281565b0-1517-437c-9cf1-7768831a0eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.244275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf902804-d824-4c13-985d-c0b63c2b7430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=088ddf9a768249a5b16e55c6e01c6e39, user_id=20b9b75717a745a18973bd43468d579a, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:34.249734 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf902804-d824-4c13-985d-c0b63c2b7430 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:34.260164 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-281565b0-1517-437c-9cf1-7768831a0eb8 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:34.261383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-281565b0-1517-437c-9cf1-7768831a0eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:34.271081 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1058/2097] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:34.271884 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281565b0-1517-437c-9cf1-7768831a0eb8 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:34.281365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281565b0-1517-437c-9cf1-7768831a0eb8 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:34.286925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b5cbe-c156-42c0-a524-9476f40aedb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:34.287523 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b5cbe-c156-42c0-a524-9476f40aedb8 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.287802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b5cbe-c156-42c0-a524-9476f40aedb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.288222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b5cbe-c156-42c0-a524-9476f40aedb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.289125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b5cbe-c156-42c0-a524-9476f40aedb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:34.289429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b5cbe-c156-42c0-a524-9476f40aedb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:34.291039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b5cbe-c156-42c0-a524-9476f40aedb8 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:34.303787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1059/2098] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:34.305087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e01d9-84e8-4d39-be12-9e027fdedd55 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.305370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e01d9-84e8-4d39-be12-9e027fdedd55 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.305640 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e01d9-84e8-4d39-be12-9e027fdedd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.326199 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d60e01d9-84e8-4d39-be12-9e027fdedd55 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:34.327467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d60e01d9-84e8-4d39-be12-9e027fdedd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:34.336291 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1060/2099] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:34.337210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e01d9-84e8-4d39-be12-9e027fdedd55 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:34.346556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e01d9-84e8-4d39-be12-9e027fdedd55 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:34.351148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b338d05-a903-46df-9d92-de829d7a57b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:34.351725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b338d05-a903-46df-9d92-de829d7a57b8 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.351999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b338d05-a903-46df-9d92-de829d7a57b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.352267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b338d05-a903-46df-9d92-de829d7a57b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.352883 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b338d05-a903-46df-9d92-de829d7a57b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:34.354825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b338d05-a903-46df-9d92-de829d7a57b8 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:34.390143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1040/2100] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:34 2022] PUT /identity/v3/projects/088ddf9a768249a5b16e55c6e01c6e39/users/20b9b75717a745a18973bd43468d579a/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:34.391183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf902804-d824-4c13-985d-c0b63c2b7430 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:34.407971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:34.454304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf902804-d824-4c13-985d-c0b63c2b7430 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:34.459678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4581c3-dc41-47b9-b43d-59ea4f9dca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:34.460704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4581c3-dc41-47b9-b43d-59ea4f9dca75 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.460704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4581c3-dc41-47b9-b43d-59ea4f9dca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.460907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4581c3-dc41-47b9-b43d-59ea4f9dca75 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.461486 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4581c3-dc41-47b9-b43d-59ea4f9dca75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:34.461778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4581c3-dc41-47b9-b43d-59ea4f9dca75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:34.463576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4581c3-dc41-47b9-b43d-59ea4f9dca75 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:34.479454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1041/2101] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:34.479454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dd2f8-eff5-41b8-8fb8-3e556ac5424a None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.479454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dd2f8-eff5-41b8-8fb8-3e556ac5424a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.479454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dd2f8-eff5-41b8-8fb8-3e556ac5424a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.481853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1061/2102] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:34 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:34.482780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b338d05-a903-46df-9d92-de829d7a57b8 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:34.509526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-128dd2f8-eff5-41b8-8fb8-3e556ac5424a None None] MFA Rules not processed for user `20b9b75717a745a18973bd43468d579a`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:34.510805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-128dd2f8-eff5-41b8-8fb8-3e556ac5424a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:34.556129 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b338d05-a903-46df-9d92-de829d7a57b8 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:34.565394 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536524e-5dbc-4c79-bee2-371c0cc407cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:34.565394 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536524e-5dbc-4c79-bee2-371c0cc407cc admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.565394 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536524e-5dbc-4c79-bee2-371c0cc407cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.565394 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536524e-5dbc-4c79-bee2-371c0cc407cc admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.565770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536524e-5dbc-4c79-bee2-371c0cc407cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:34.568209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536524e-5dbc-4c79-bee2-371c0cc407cc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:34.586053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1042/2103] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:34.586986 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128dd2f8-eff5-41b8-8fb8-3e556ac5424a None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:34.599419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128dd2f8-eff5-41b8-8fb8-3e556ac5424a None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:34.604026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5831857-80c6-4f5c-8cfe-17317dbb4a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:34.604622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5831857-80c6-4f5c-8cfe-17317dbb4a5d admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.604899 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5831857-80c6-4f5c-8cfe-17317dbb4a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.605170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5831857-80c6-4f5c-8cfe-17317dbb4a5d admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.605824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5831857-80c6-4f5c-8cfe-17317dbb4a5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:34.607756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5831857-80c6-4f5c-8cfe-17317dbb4a5d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:34.705964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1043/2104] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:34 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:34.706549 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1062/2105] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:34 2022] POST /identity/v3/projects => generated 387 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:34.707435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536524e-5dbc-4c79-bee2-371c0cc407cc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:34.710568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5831857-80c6-4f5c-8cfe-17317dbb4a5d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:34.710684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b536524e-5dbc-4c79-bee2-371c0cc407cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:34.712362 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a5831857-80c6-4f5c-8cfe-17317dbb4a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:34.780058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5831857-80c6-4f5c-8cfe-17317dbb4a5d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:34.780699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536524e-5dbc-4c79-bee2-371c0cc407cc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:34.787289 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:34.788079 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.796214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.796214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] PATH_INFO: `/v3/projects/199ddd039732406795849fb6044a4f72` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.796214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] RBAC: Authorizing `identity:delete_project(project_id=199ddd039732406795849fb6044a4f72)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:34.796214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:34.839188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032d379-8904-4f57-924c-762d5c35ec76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:34.842070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032d379-8904-4f57-924c-762d5c35ec76 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.842070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032d379-8904-4f57-924c-762d5c35ec76 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.842070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032d379-8904-4f57-924c-762d5c35ec76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.899552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032d379-8904-4f57-924c-762d5c35ec76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:34.901013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032d379-8904-4f57-924c-762d5c35ec76 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:34.905009 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '199ddd039732406795849fb6044a4f72', 'request_id': 'req-b7069b38-3e9e-4811-baf1-fa0ad47e827f'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:34.911860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1044/2106] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:34.913914 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032d379-8904-4f57-924c-762d5c35ec76 service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:34.928519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '199ddd039732406795849fb6044a4f72', 'request_id': 'req-b7069b38-3e9e-4811-baf1-fa0ad47e827f'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:34.928772 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032d379-8904-4f57-924c-762d5c35ec76 service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:34.988872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217b498-d17e-4728-9da5-86678219002c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:34.989482 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217b498-d17e-4728-9da5-86678219002c admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:34.989763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217b498-d17e-4728-9da5-86678219002c admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:34.990037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217b498-d17e-4728-9da5-86678219002c admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:34.990418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '199ddd039732406795849fb6044a4f72', 'request_id': 'req-b7069b38-3e9e-4811-baf1-fa0ad47e827f'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:34.990706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217b498-d17e-4728-9da5-86678219002c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:34.992293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217b498-d17e-4728-9da5-86678219002c admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:35.002820 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1045/2107] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:35.003624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217b498-d17e-4728-9da5-86678219002c admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:35.005933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] The token cache is being invalidate because project 199ddd039732406795849fb6044a4f72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:35.006226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:35.006904 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:35.008383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1063/2108] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:34 2022] DELETE /identity/v3/projects/199ddd039732406795849fb6044a4f72 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:35.009317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:35.011173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:35.014270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217b498-d17e-4728-9da5-86678219002c admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:35.018959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199c33f-0930-455b-9b45-73433454e915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:35.020157 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199c33f-0930-455b-9b45-73433454e915 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.020554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199c33f-0930-455b-9b45-73433454e915 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.020839 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199c33f-0930-455b-9b45-73433454e915 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.021542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199c33f-0930-455b-9b45-73433454e915 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:35.024531 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199c33f-0930-455b-9b45-73433454e915 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:35.026645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7069b38-3e9e-4811-baf1-fa0ad47e827f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:35.032217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c86a13-61d9-440e-a93b-71c54a67ae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:35.032830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c86a13-61d9-440e-a93b-71c54a67ae44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.033098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c86a13-61d9-440e-a93b-71c54a67ae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.033372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c86a13-61d9-440e-a93b-71c54a67ae44 admin admin] PATH_INFO: `/v3/users/20b9b75717a745a18973bd43468d579a` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.034528 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c86a13-61d9-440e-a93b-71c54a67ae44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20b9b75717a745a18973bd43468d579a)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:35.036559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c86a13-61d9-440e-a93b-71c54a67ae44 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:35.141810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1046/2109] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:35 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:35.142565 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199c33f-0930-455b-9b45-73433454e915 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:35.145562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5199c33f-0930-455b-9b45-73433454e915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:35.160916 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199c33f-0930-455b-9b45-73433454e915 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:35.179459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10858d16-e002-488d-8e7e-b0cac87a4d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:35.180087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10858d16-e002-488d-8e7e-b0cac87a4d39 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.180434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10858d16-e002-488d-8e7e-b0cac87a4d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.180625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10858d16-e002-488d-8e7e-b0cac87a4d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.181408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10858d16-e002-488d-8e7e-b0cac87a4d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:35.183318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10858d16-e002-488d-8e7e-b0cac87a4d39 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:35.195482 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1047/2110] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:35.196710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04949457-6ba7-4a24-a65e-d46293fd18d5 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.196991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04949457-6ba7-4a24-a65e-d46293fd18d5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.197263 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04949457-6ba7-4a24-a65e-d46293fd18d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.218281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-04949457-6ba7-4a24-a65e-d46293fd18d5 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:35.219673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-04949457-6ba7-4a24-a65e-d46293fd18d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:35.228103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c3c86a13-61d9-440e-a93b-71c54a67ae44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20b9b75717a745a18973bd43468d579a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:35.238205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1048/2111] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:35.239448 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04949457-6ba7-4a24-a65e-d46293fd18d5 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:35.240859 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-04949457-6ba7-4a24-a65e-d46293fd18d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:35.253069 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04949457-6ba7-4a24-a65e-d46293fd18d5 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:35.258940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117d7b5-c798-4f2d-aec3-144ca56e1d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:35.259549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117d7b5-c798-4f2d-aec3-144ca56e1d4c admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.259832 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117d7b5-c798-4f2d-aec3-144ca56e1d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.260101 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117d7b5-c798-4f2d-aec3-144ca56e1d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.261631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117d7b5-c798-4f2d-aec3-144ca56e1d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:35.262627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117d7b5-c798-4f2d-aec3-144ca56e1d4c admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:35.275077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1049/2112] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:35.275077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117d7b5-c798-4f2d-aec3-144ca56e1d4c admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:35.287724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117d7b5-c798-4f2d-aec3-144ca56e1d4c admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:35.292466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234253e-5b0e-4052-9b65-80ad0187d72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:35.293113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234253e-5b0e-4052-9b65-80ad0187d72a admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.293393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234253e-5b0e-4052-9b65-80ad0187d72a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.293686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234253e-5b0e-4052-9b65-80ad0187d72a admin admin] PATH_INFO: `/v3/projects/c8c7a97f6135463b924e15fec21f6b43/users/feea737a0e5f44c3a97bd9a859f71e67/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.311397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234253e-5b0e-4052-9b65-80ad0187d72a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c7a97f6135463b924e15fec21f6b43, user_id=feea737a0e5f44c3a97bd9a859f71e67, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:35.314843 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234253e-5b0e-4052-9b65-80ad0187d72a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:35.354638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c3c86a13-61d9-440e-a93b-71c54a67ae44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20b9b75717a745a18973bd43468d579a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:35.355287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c3c86a13-61d9-440e-a93b-71c54a67ae44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20b9b75717a745a18973bd43468d579a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:35.355997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c3c86a13-61d9-440e-a93b-71c54a67ae44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20b9b75717a745a18973bd43468d579a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:35.369729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c3c86a13-61d9-440e-a93b-71c54a67ae44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20b9b75717a745a18973bd43468d579a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:35.381998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1064/2113] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:35 2022] DELETE /identity/v3/users/20b9b75717a745a18973bd43468d579a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:35.383259 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2d4be-7ab2-40d0-b5c9-6bc3a4c54259 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.383539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2d4be-7ab2-40d0-b5c9-6bc3a4c54259 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.383814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2d4be-7ab2-40d0-b5c9-6bc3a4c54259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.403781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-8bb2d4be-7ab2-40d0-b5c9-6bc3a4c54259 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:35.405032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8bb2d4be-7ab2-40d0-b5c9-6bc3a4c54259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:35.423111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1050/2114] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:35 2022] PUT /identity/v3/projects/c8c7a97f6135463b924e15fec21f6b43/users/feea737a0e5f44c3a97bd9a859f71e67/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:35.423702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234253e-5b0e-4052-9b65-80ad0187d72a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:35.425749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d234253e-5b0e-4052-9b65-80ad0187d72a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:35.448321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:35.472780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1065/2115] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:35.475045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca765e0f-27d3-42a6-88d0-d463434f2572 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.475334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca765e0f-27d3-42a6-88d0-d463434f2572 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.475612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca765e0f-27d3-42a6-88d0-d463434f2572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.495072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-ca765e0f-27d3-42a6-88d0-d463434f2572 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:35.496289 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ca765e0f-27d3-42a6-88d0-d463434f2572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:35.512226 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234253e-5b0e-4052-9b65-80ad0187d72a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:35.517589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e318962-3595-4575-a25a-eea70ef1218e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:35.518388 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e318962-3595-4575-a25a-eea70ef1218e admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.518853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e318962-3595-4575-a25a-eea70ef1218e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.519204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e318962-3595-4575-a25a-eea70ef1218e admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.520616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e318962-3595-4575-a25a-eea70ef1218e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:35.522834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e318962-3595-4575-a25a-eea70ef1218e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:35.535802 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1051/2116] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:35.539777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676b006-d517-4c55-a241-7a6b150869e5 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.539777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676b006-d517-4c55-a241-7a6b150869e5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.539777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676b006-d517-4c55-a241-7a6b150869e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.578805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1066/2117] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:35.579835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca765e0f-27d3-42a6-88d0-d463434f2572 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:35.584590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-4676b006-d517-4c55-a241-7a6b150869e5 None None] MFA Rules not processed for user `feea737a0e5f44c3a97bd9a859f71e67`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:35.585987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-4676b006-d517-4c55-a241-7a6b150869e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:35.593634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca765e0f-27d3-42a6-88d0-d463434f2572 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:35.598810 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe5f57-c863-43d9-a6ac-429aa78b86a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:35.599578 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe5f57-c863-43d9-a6ac-429aa78b86a1 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.599864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe5f57-c863-43d9-a6ac-429aa78b86a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.600140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe5f57-c863-43d9-a6ac-429aa78b86a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.600808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe5f57-c863-43d9-a6ac-429aa78b86a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:35.601114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe5f57-c863-43d9-a6ac-429aa78b86a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:35.602808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe5f57-c863-43d9-a6ac-429aa78b86a1 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:35.607953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:35.624700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1067/2118] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:35.624700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe5f57-c863-43d9-a6ac-429aa78b86a1 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:35.634000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe5f57-c863-43d9-a6ac-429aa78b86a1 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:35.639406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a632ec-ea08-4fab-9dd8-87cea43452ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:35.640129 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a632ec-ea08-4fab-9dd8-87cea43452ed admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.649386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a632ec-ea08-4fab-9dd8-87cea43452ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.649386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a632ec-ea08-4fab-9dd8-87cea43452ed admin admin] PATH_INFO: `/v3/projects/0c8f9106e3da46ec8661ed5c920a75bd/users/6d1d1765412b42d8a33d66ad85811d36/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.656037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a632ec-ea08-4fab-9dd8-87cea43452ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c8f9106e3da46ec8661ed5c920a75bd, user_id=6d1d1765412b42d8a33d66ad85811d36, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:35.658699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a632ec-ea08-4fab-9dd8-87cea43452ed admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:35.675313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1052/2119] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:35.676713 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676b006-d517-4c55-a241-7a6b150869e5 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:35.678081 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-4676b006-d517-4c55-a241-7a6b150869e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:35.735469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676b006-d517-4c55-a241-7a6b150869e5 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:35.739762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a08f6-db21-4dfe-aff4-4ebfcdd05c3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:35.740514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a08f6-db21-4dfe-aff4-4ebfcdd05c3e service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.740770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a08f6-db21-4dfe-aff4-4ebfcdd05c3e service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.741044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a08f6-db21-4dfe-aff4-4ebfcdd05c3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.743971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1068/2120] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:35 2022] PUT /identity/v3/projects/0c8f9106e3da46ec8661ed5c920a75bd/users/6d1d1765412b42d8a33d66ad85811d36/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:35.746906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a632ec-ea08-4fab-9dd8-87cea43452ed admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:35.750546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a08f6-db21-4dfe-aff4-4ebfcdd05c3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:35.752048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a08f6-db21-4dfe-aff4-4ebfcdd05c3e service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:35.762053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1053/2121] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:35.762903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a08f6-db21-4dfe-aff4-4ebfcdd05c3e service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:35.764295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c65a08f6-db21-4dfe-aff4-4ebfcdd05c3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:35.826666 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a08f6-db21-4dfe-aff4-4ebfcdd05c3e service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:35.830420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a632ec-ea08-4fab-9dd8-87cea43452ed admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:35.836521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a07204-db87-4967-b8ae-7d860e09fc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:35.837157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a07204-db87-4967-b8ae-7d860e09fc85 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.837439 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a07204-db87-4967-b8ae-7d860e09fc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.837713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a07204-db87-4967-b8ae-7d860e09fc85 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.838614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a07204-db87-4967-b8ae-7d860e09fc85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:35.840539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a07204-db87-4967-b8ae-7d860e09fc85 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:35.880055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0323848-20d5-441c-b086-75a40ff4ffcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:35.880652 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0323848-20d5-441c-b086-75a40ff4ffcb service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.880927 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0323848-20d5-441c-b086-75a40ff4ffcb service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.881200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0323848-20d5-441c-b086-75a40ff4ffcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.936062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0323848-20d5-441c-b086-75a40ff4ffcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:35.937489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0323848-20d5-441c-b086-75a40ff4ffcb service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:35.947279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1054/2122] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:35.950080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fc474-8dd9-4ed2-9054-8fb10ab0bceb None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:35.950372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fc474-8dd9-4ed2-9054-8fb10ab0bceb None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:35.950669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fc474-8dd9-4ed2-9054-8fb10ab0bceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:35.954120 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1069/2123] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:35 2022] POST /identity/v3/projects => generated 383 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:35.960103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a07204-db87-4967-b8ae-7d860e09fc85 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:35.970466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=81,keystone:INSERT=5,keystone:DELETE=14 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:35.988247 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-2c9fc474-8dd9-4ed2-9054-8fb10ab0bceb None None] MFA Rules not processed for user `6d1d1765412b42d8a33d66ad85811d36`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:35.991956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2c9fc474-8dd9-4ed2-9054-8fb10ab0bceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:36.059381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a07204-db87-4967-b8ae-7d860e09fc85 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:36.064248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e216b81-f4c7-44a0-a668-41ea2574c6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:36.064867 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e216b81-f4c7-44a0-a668-41ea2574c6ca admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:36.065147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e216b81-f4c7-44a0-a668-41ea2574c6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:36.065386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e216b81-f4c7-44a0-a668-41ea2574c6ca admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:36.066046 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e216b81-f4c7-44a0-a668-41ea2574c6ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:36.068973 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e216b81-f4c7-44a0-a668-41ea2574c6ca admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:36.113230 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1055/2124] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:36.118768 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9fc474-8dd9-4ed2-9054-8fb10ab0bceb None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:36.121740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:36.137464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9fc474-8dd9-4ed2-9054-8fb10ab0bceb None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:36.142225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a87c57-f909-4c4e-b942-b84c97c4d906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:36.143070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a87c57-f909-4c4e-b942-b84c97c4d906 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:36.143396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a87c57-f909-4c4e-b942-b84c97c4d906 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:36.143899 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a87c57-f909-4c4e-b942-b84c97c4d906 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:36.144576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a87c57-f909-4c4e-b942-b84c97c4d906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:36.146798 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a87c57-f909-4c4e-b942-b84c97c4d906 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:36.150508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1070/2125] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:35 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:36.154280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e216b81-f4c7-44a0-a668-41ea2574c6ca admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:36.164063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e216b81-f4c7-44a0-a668-41ea2574c6ca admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:36.168688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3208bf6-8c53-4910-a5ec-48a594249ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:36.169268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3208bf6-8c53-4910-a5ec-48a594249ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:36.169549 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3208bf6-8c53-4910-a5ec-48a594249ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:36.169869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3208bf6-8c53-4910-a5ec-48a594249ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:36.170443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3208bf6-8c53-4910-a5ec-48a594249ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:36.172091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3208bf6-8c53-4910-a5ec-48a594249ef6 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:36.183102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1071/2126] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:36.187091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3208bf6-8c53-4910-a5ec-48a594249ef6 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:36.197264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3208bf6-8c53-4910-a5ec-48a594249ef6 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:36.201882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b963d32-248c-4fbf-9e1e-20ceb2866f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:36.202464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b963d32-248c-4fbf-9e1e-20ceb2866f56 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:36.203079 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b963d32-248c-4fbf-9e1e-20ceb2866f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:36.203423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b963d32-248c-4fbf-9e1e-20ceb2866f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:36.204696 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b963d32-248c-4fbf-9e1e-20ceb2866f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:36.206413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b963d32-248c-4fbf-9e1e-20ceb2866f56 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:36.217799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1072/2127] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:36.220329 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b963d32-248c-4fbf-9e1e-20ceb2866f56 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:36.236120 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b963d32-248c-4fbf-9e1e-20ceb2866f56 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:36.243726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facef1b4-9608-4531-b401-2f0e2bc69313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:36.244330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facef1b4-9608-4531-b401-2f0e2bc69313 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:36.244618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facef1b4-9608-4531-b401-2f0e2bc69313 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:36.244896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facef1b4-9608-4531-b401-2f0e2bc69313 admin admin] PATH_INFO: `/v3/projects/d1809431125f4ca3bbcd4e84992e340e/users/7817e876611b42498db2fa5304c89a75/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:36.266081 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facef1b4-9608-4531-b401-2f0e2bc69313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1809431125f4ca3bbcd4e84992e340e, user_id=7817e876611b42498db2fa5304c89a75, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:36.268815 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facef1b4-9608-4531-b401-2f0e2bc69313 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:36.283622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1056/2128] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:36 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:57:36.284466 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a87c57-f909-4c4e-b942-b84c97c4d906 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:36.286227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f0a87c57-f909-4c4e-b942-b84c97c4d906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:36.351408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a87c57-f909-4c4e-b942-b84c97c4d906 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:36.352012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1073/2129] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:36 2022] PUT /identity/v3/projects/d1809431125f4ca3bbcd4e84992e340e/users/7817e876611b42498db2fa5304c89a75/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:36.352603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facef1b4-9608-4531-b401-2f0e2bc69313 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:36.398753 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:36.418858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facef1b4-9608-4531-b401-2f0e2bc69313 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:36.424324 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2afdf9-f4aa-46e5-b887-e0ae201a2755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:36.425102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2afdf9-f4aa-46e5-b887-e0ae201a2755 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:36.425316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2afdf9-f4aa-46e5-b887-e0ae201a2755 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:36.425661 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2afdf9-f4aa-46e5-b887-e0ae201a2755 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:36.426814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2afdf9-f4aa-46e5-b887-e0ae201a2755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:36.427570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:36.428976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:36.429086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2afdf9-f4aa-46e5-b887-e0ae201a2755 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:36.429162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:36.429431 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] PATH_INFO: `/v3/projects/088ddf9a768249a5b16e55c6e01c6e39` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:36.430638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] RBAC: Authorizing `identity:delete_project(project_id=088ddf9a768249a5b16e55c6e01c6e39)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:36.432887 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:36.542984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1074/2130] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:36 2022] POST /identity/v3/users => generated 299 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:36.544248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3320b10-d3e8-4f30-8286-ed0700970a00 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:36.544534 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3320b10-d3e8-4f30-8286-ed0700970a00 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:36.544811 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3320b10-d3e8-4f30-8286-ed0700970a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:36.571848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-f3320b10-d3e8-4f30-8286-ed0700970a00 None None] MFA Rules not processed for user `7817e876611b42498db2fa5304c89a75`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:36.573049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f3320b10-d3e8-4f30-8286-ed0700970a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:36.603234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '088ddf9a768249a5b16e55c6e01c6e39', 'request_id': 'req-e2c60179-1e53-4981-9d60-418d6772a208'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:36.609646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '088ddf9a768249a5b16e55c6e01c6e39', 'request_id': 'req-e2c60179-1e53-4981-9d60-418d6772a208'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:36.648740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1075/2131] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:36.649608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3320b10-d3e8-4f30-8286-ed0700970a00 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:36.668173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3320b10-d3e8-4f30-8286-ed0700970a00 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:36.688158 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '088ddf9a768249a5b16e55c6e01c6e39', 'request_id': 'req-e2c60179-1e53-4981-9d60-418d6772a208'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:36.696044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:36.701644 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] The token cache is being invalidate because project 088ddf9a768249a5b16e55c6e01c6e39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:36.701939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:36.703659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:36.705197 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1057/2132] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:36 2022] DELETE /identity/v3/projects/088ddf9a768249a5b16e55c6e01c6e39 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:36.706588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:36.711795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:36.780567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f7a98-7fb8-4d0a-9b81-2f056f6958d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:36.781347 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f7a98-7fb8-4d0a-9b81-2f056f6958d2 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:36.781625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f7a98-7fb8-4d0a-9b81-2f056f6958d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:36.781896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f7a98-7fb8-4d0a-9b81-2f056f6958d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:36.782585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f7a98-7fb8-4d0a-9b81-2f056f6958d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:36.785256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f7a98-7fb8-4d0a-9b81-2f056f6958d2 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:36.803859 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c60179-1e53-4981-9d60-418d6772a208 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:36.807595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1076/2133] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:36.807595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f7a98-7fb8-4d0a-9b81-2f056f6958d2 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:36.808725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-3b8f7a98-7fb8-4d0a-9b81-2f056f6958d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:36.814766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ddc2e-3a1c-4236-b61b-eda1b7ac58c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:36.814766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ddc2e-3a1c-4236-b61b-eda1b7ac58c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:36.814766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ddc2e-3a1c-4236-b61b-eda1b7ac58c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:36.814766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ddc2e-3a1c-4236-b61b-eda1b7ac58c5 admin admin] PATH_INFO: `/v3/users/9e642f25cca546f996d23fb579147c7b` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:36.815767 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ddc2e-3a1c-4236-b61b-eda1b7ac58c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e642f25cca546f996d23fb579147c7b)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:36.819400 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ddc2e-3a1c-4236-b61b-eda1b7ac58c5 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:36.824021 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f7a98-7fb8-4d0a-9b81-2f056f6958d2 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:36.829974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269da5ae-8189-4f59-b070-faf284349da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:36.830649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269da5ae-8189-4f59-b070-faf284349da3 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:36.831191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269da5ae-8189-4f59-b070-faf284349da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:36.831191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269da5ae-8189-4f59-b070-faf284349da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:36.831957 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269da5ae-8189-4f59-b070-faf284349da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:36.835642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269da5ae-8189-4f59-b070-faf284349da3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:36.949736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1077/2134] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:36 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:36.949736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9b97c-1e63-4aba-8c1b-45b05c129ace None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:36.949736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9b97c-1e63-4aba-8c1b-45b05c129ace None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:36.949736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9b97c-1e63-4aba-8c1b-45b05c129ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:36.968580 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-81e9b97c-1e63-4aba-8c1b-45b05c129ace None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:36.969872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-81e9b97c-1e63-4aba-8c1b-45b05c129ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:37.032804 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1078/2135] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:37.039532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9b97c-1e63-4aba-8c1b-45b05c129ace None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:37.039532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-81e9b97c-1e63-4aba-8c1b-45b05c129ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:37.048255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9b97c-1e63-4aba-8c1b-45b05c129ace None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:37.053941 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:37.054100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af411c-c402-43a7-9c67-c2c163fab90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:37.054886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af411c-c402-43a7-9c67-c2c163fab90a admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.060222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af411c-c402-43a7-9c67-c2c163fab90a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.060222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af411c-c402-43a7-9c67-c2c163fab90a admin admin] PATH_INFO: `/v3/projects/35f3224115fb46ac956b15f733fdacec/users/eea04a5c72aa499f95cec718147fa55f/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.071022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af411c-c402-43a7-9c67-c2c163fab90a admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f3224115fb46ac956b15f733fdacec, user_id=eea04a5c72aa499f95cec718147fa55f, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:37.073679 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af411c-c402-43a7-9c67-c2c163fab90a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:37.094450 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-097ddc2e-3a1c-4236-b61b-eda1b7ac58c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e642f25cca546f996d23fb579147c7b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:37.143245 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1079/2136] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:37 2022] PUT /identity/v3/projects/35f3224115fb46ac956b15f733fdacec/users/eea04a5c72aa499f95cec718147fa55f/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:37.144069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af411c-c402-43a7-9c67-c2c163fab90a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:37.194237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-097ddc2e-3a1c-4236-b61b-eda1b7ac58c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e642f25cca546f996d23fb579147c7b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:37.194920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-097ddc2e-3a1c-4236-b61b-eda1b7ac58c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e642f25cca546f996d23fb579147c7b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:37.195547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-097ddc2e-3a1c-4236-b61b-eda1b7ac58c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e642f25cca546f996d23fb579147c7b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:37.203942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-097ddc2e-3a1c-4236-b61b-eda1b7ac58c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e642f25cca546f996d23fb579147c7b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:37.209307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af411c-c402-43a7-9c67-c2c163fab90a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:37.214846 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36160af-ddfd-4380-ac9d-2141fbb12d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:37.215471 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36160af-ddfd-4380-ac9d-2141fbb12d07 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.215754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36160af-ddfd-4380-ac9d-2141fbb12d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.216031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36160af-ddfd-4380-ac9d-2141fbb12d07 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.216678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36160af-ddfd-4380-ac9d-2141fbb12d07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:37.218784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36160af-ddfd-4380-ac9d-2141fbb12d07 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:37.225989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1058/2137] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:36 2022] DELETE /identity/v3/users/9e642f25cca546f996d23fb579147c7b => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:37.230515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dda967-478f-4059-96f5-800d0d271a3e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.230715 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dda967-478f-4059-96f5-800d0d271a3e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.230924 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dda967-478f-4059-96f5-800d0d271a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.251867 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-29dda967-478f-4059-96f5-800d0d271a3e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:37.253104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-29dda967-478f-4059-96f5-800d0d271a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:37.317374 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1059/2138] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:37.319003 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dda967-478f-4059-96f5-800d0d271a3e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:37.321920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-29dda967-478f-4059-96f5-800d0d271a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:37.334383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dda967-478f-4059-96f5-800d0d271a3e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:37.339096 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92786529-1d9c-470e-be33-894125ab4a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:37.339738 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92786529-1d9c-470e-be33-894125ab4a36 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.340019 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92786529-1d9c-470e-be33-894125ab4a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.340293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92786529-1d9c-470e-be33-894125ab4a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.340940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92786529-1d9c-470e-be33-894125ab4a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:37.342584 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92786529-1d9c-470e-be33-894125ab4a36 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:37.354651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1060/2139] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:37.359956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f628ce-74f9-4d12-bbe5-44956bcd98f8 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.360086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1080/2140] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:37 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:37.360180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f628ce-74f9-4d12-bbe5-44956bcd98f8 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.360180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f628ce-74f9-4d12-bbe5-44956bcd98f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.360333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36160af-ddfd-4380-ac9d-2141fbb12d07 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:37.372569 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36160af-ddfd-4380-ac9d-2141fbb12d07 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:37.378055 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0e51c-1695-49af-9bac-92f510f91950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:37.378849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0e51c-1695-49af-9bac-92f510f91950 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.379130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0e51c-1695-49af-9bac-92f510f91950 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.379290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-62f628ce-74f9-4d12-bbe5-44956bcd98f8 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:37.382671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0e51c-1695-49af-9bac-92f510f91950 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.382671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0e51c-1695-49af-9bac-92f510f91950 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:37.382840 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-62f628ce-74f9-4d12-bbe5-44956bcd98f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:37.382912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0e51c-1695-49af-9bac-92f510f91950 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:37.382912 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0e51c-1695-49af-9bac-92f510f91950 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:37.391947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1061/2141] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:37.391947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f628ce-74f9-4d12-bbe5-44956bcd98f8 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:37.391947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-62f628ce-74f9-4d12-bbe5-44956bcd98f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:37.403555 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f628ce-74f9-4d12-bbe5-44956bcd98f8 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:37.404762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1081/2142] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:37.411157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0e51c-1695-49af-9bac-92f510f91950 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:37.411268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d0492-28e5-4d76-a99a-55ad659127a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:37.411268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d0492-28e5-4d76-a99a-55ad659127a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.411268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d0492-28e5-4d76-a99a-55ad659127a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.411268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d0492-28e5-4d76-a99a-55ad659127a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.411909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d0492-28e5-4d76-a99a-55ad659127a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:37.413736 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d0492-28e5-4d76-a99a-55ad659127a0 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:37.419504 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0e51c-1695-49af-9bac-92f510f91950 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:37.424285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115250c-8f47-4351-bf91-933da6c774ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:37.425112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115250c-8f47-4351-bf91-933da6c774ee admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.425270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1062/2143] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:37.425888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115250c-8f47-4351-bf91-933da6c774ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.426875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d0492-28e5-4d76-a99a-55ad659127a0 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:37.427052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115250c-8f47-4351-bf91-933da6c774ee admin admin] PATH_INFO: `/v3/projects/35f3224115fb46ac956b15f733fdacec/users/eea04a5c72aa499f95cec718147fa55f/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.430542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-622d0492-28e5-4d76-a99a-55ad659127a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:37.430872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115250c-8f47-4351-bf91-933da6c774ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f3224115fb46ac956b15f733fdacec, user_id=eea04a5c72aa499f95cec718147fa55f, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:37.433636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115250c-8f47-4351-bf91-933da6c774ee admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:37.444678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1082/2144] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:37 2022] PUT /identity/v3/projects/35f3224115fb46ac956b15f733fdacec/users/eea04a5c72aa499f95cec718147fa55f/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:37.445925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115250c-8f47-4351-bf91-933da6c774ee admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:37.489010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:37.502677 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d0492-28e5-4d76-a99a-55ad659127a0 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:37.522081 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115250c-8f47-4351-bf91-933da6c774ee admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:37.527054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd550c3-28a6-4603-852f-538a2d0f8182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:37.527664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd550c3-28a6-4603-852f-538a2d0f8182 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.527939 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd550c3-28a6-4603-852f-538a2d0f8182 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.528261 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd550c3-28a6-4603-852f-538a2d0f8182 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.528990 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd550c3-28a6-4603-852f-538a2d0f8182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:37.530941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd550c3-28a6-4603-852f-538a2d0f8182 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:37.555926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6ba78-f8ee-4ef1-938d-8cab01acd039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:37.556544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6ba78-f8ee-4ef1-938d-8cab01acd039 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.556822 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6ba78-f8ee-4ef1-938d-8cab01acd039 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.557086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6ba78-f8ee-4ef1-938d-8cab01acd039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.566339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6ba78-f8ee-4ef1-938d-8cab01acd039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:37.569395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6ba78-f8ee-4ef1-938d-8cab01acd039 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:37.580016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1063/2145] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:37.580863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6ba78-f8ee-4ef1-938d-8cab01acd039 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:37.591911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6ba78-f8ee-4ef1-938d-8cab01acd039 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:37.597160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60406e36-9874-460b-8aaa-59e8b1ad13bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:37.597808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60406e36-9874-460b-8aaa-59e8b1ad13bc admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.598168 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60406e36-9874-460b-8aaa-59e8b1ad13bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.598777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60406e36-9874-460b-8aaa-59e8b1ad13bc admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.599734 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60406e36-9874-460b-8aaa-59e8b1ad13bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:37.601397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60406e36-9874-460b-8aaa-59e8b1ad13bc admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:37.606431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1083/2146] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:37 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:37.606431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a421d-22d6-434a-babf-70ac53052860 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.606431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a421d-22d6-434a-babf-70ac53052860 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.606779 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a421d-22d6-434a-babf-70ac53052860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.613150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1064/2147] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:37.615612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60406e36-9874-460b-8aaa-59e8b1ad13bc admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:37.639863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60406e36-9874-460b-8aaa-59e8b1ad13bc admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:37.666998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-919a421d-22d6-434a-babf-70ac53052860 None None] MFA Rules not processed for user `eea04a5c72aa499f95cec718147fa55f`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:37.699652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-919a421d-22d6-434a-babf-70ac53052860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:37.722134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4d765-51a1-43fe-8a86-b79358e3da1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:37.723045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4d765-51a1-43fe-8a86-b79358e3da1d admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.723355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4d765-51a1-43fe-8a86-b79358e3da1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.723652 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4d765-51a1-43fe-8a86-b79358e3da1d admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.724302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4d765-51a1-43fe-8a86-b79358e3da1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:37.726282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4d765-51a1-43fe-8a86-b79358e3da1d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:37.763675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1084/2148] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:37.767355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a421d-22d6-434a-babf-70ac53052860 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:37.776266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a421d-22d6-434a-babf-70ac53052860 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:37.780977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc82bc-a14e-4444-8dbf-3415153b2914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:37.781666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc82bc-a14e-4444-8dbf-3415153b2914 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:37.781941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc82bc-a14e-4444-8dbf-3415153b2914 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:37.782210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc82bc-a14e-4444-8dbf-3415153b2914 admin admin] PATH_INFO: `/v3/projects/2246860985904ff8a0a6afdd06e8bef5/users/2fc6a725206d41bf9bb36da5ee7ec707/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:37.801001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc82bc-a14e-4444-8dbf-3415153b2914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2246860985904ff8a0a6afdd06e8bef5, user_id=2fc6a725206d41bf9bb36da5ee7ec707, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:37.805258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc82bc-a14e-4444-8dbf-3415153b2914 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:38.013577 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1085/2149] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:37 2022] PUT /identity/v3/projects/2246860985904ff8a0a6afdd06e8bef5/users/2fc6a725206d41bf9bb36da5ee7ec707/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:38.019260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc82bc-a14e-4444-8dbf-3415153b2914 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:38.019406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1065/2150] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:37 2022] POST /identity/v3/users => generated 288 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:38.019497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-35fc82bc-a14e-4444-8dbf-3415153b2914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:38.024808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4d765-51a1-43fe-8a86-b79358e3da1d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:38.050243 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:38.056712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4d765-51a1-43fe-8a86-b79358e3da1d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:38.129865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:38.134203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc82bc-a14e-4444-8dbf-3415153b2914 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:38.167517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a6bd1-9ac2-4261-a918-77497c545456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:38.167517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a6bd1-9ac2-4261-a918-77497c545456 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:38.167517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a6bd1-9ac2-4261-a918-77497c545456 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:38.167517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a6bd1-9ac2-4261-a918-77497c545456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:38.169690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9589c3b-d2f2-4da9-b53f-24707fe82b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:38.169690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9589c3b-d2f2-4da9-b53f-24707fe82b70 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:38.169690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9589c3b-d2f2-4da9-b53f-24707fe82b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:38.169690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9589c3b-d2f2-4da9-b53f-24707fe82b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:38.170857 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9589c3b-d2f2-4da9-b53f-24707fe82b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:38.170857 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9589c3b-d2f2-4da9-b53f-24707fe82b70 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:38.199306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1066/2151] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:38.199306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c92bd-5ef5-4ad4-9a71-3a916772fe24 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:38.199306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c92bd-5ef5-4ad4-9a71-3a916772fe24 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:38.199306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c92bd-5ef5-4ad4-9a71-3a916772fe24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:38.265101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a6bd1-9ac2-4261-a918-77497c545456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:38.265101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a6bd1-9ac2-4261-a918-77497c545456 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:38.280377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-c11c92bd-5ef5-4ad4-9a71-3a916772fe24 None None] MFA Rules not processed for user `2fc6a725206d41bf9bb36da5ee7ec707`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:38.281611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c11c92bd-5ef5-4ad4-9a71-3a916772fe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:38.294469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1086/2152] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:38.295606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a6bd1-9ac2-4261-a918-77497c545456 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:38.308658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a6bd1-9ac2-4261-a918-77497c545456 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:38.319226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1e891-b1b5-4a0f-9e98-969d6b9551e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:38.319734 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1e891-b1b5-4a0f-9e98-969d6b9551e2 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:38.320150 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1e891-b1b5-4a0f-9e98-969d6b9551e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:38.320425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1e891-b1b5-4a0f-9e98-969d6b9551e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:38.321264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1e891-b1b5-4a0f-9e98-969d6b9551e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:38.326055 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1e891-b1b5-4a0f-9e98-969d6b9551e2 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:38.337462 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1087/2153] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:38.344464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1e891-b1b5-4a0f-9e98-969d6b9551e2 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:38.356025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1e891-b1b5-4a0f-9e98-969d6b9551e2 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:38.365326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d5fb-e012-4d76-b0fe-534732c521a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:38.365326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d5fb-e012-4d76-b0fe-534732c521a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:38.365326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d5fb-e012-4d76-b0fe-534732c521a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:38.365326 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d5fb-e012-4d76-b0fe-534732c521a3 admin admin] PATH_INFO: `/v3/projects/b049acdc82024f5480b9804239437d85/users/1e77b5549d44484b9be2bb3d7c7dd4df/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:38.395616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5d5fb-e012-4d76-b0fe-534732c521a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b049acdc82024f5480b9804239437d85, user_id=1e77b5549d44484b9be2bb3d7c7dd4df, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:38.395616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5d5fb-e012-4d76-b0fe-534732c521a3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:38.409452 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1067/2154] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:38.409452 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c92bd-5ef5-4ad4-9a71-3a916772fe24 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:38.412896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c11c92bd-5ef5-4ad4-9a71-3a916772fe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:38.428330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c92bd-5ef5-4ad4-9a71-3a916772fe24 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:38.432977 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83424433-9d18-468e-bbe9-a0d199f77d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:38.433603 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83424433-9d18-468e-bbe9-a0d199f77d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:38.433910 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83424433-9d18-468e-bbe9-a0d199f77d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:38.434179 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83424433-9d18-468e-bbe9-a0d199f77d5d admin admin] PATH_INFO: `/v3/projects/309d94b39e8c403eb5f36cf2ccd9c55d` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:38.435355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83424433-9d18-468e-bbe9-a0d199f77d5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=309d94b39e8c403eb5f36cf2ccd9c55d)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:38.437241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83424433-9d18-468e-bbe9-a0d199f77d5d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:38.510378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1088/2155] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:38 2022] PUT /identity/v3/projects/b049acdc82024f5480b9804239437d85/users/1e77b5549d44484b9be2bb3d7c7dd4df/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:38.511303 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d5fb-e012-4d76-b0fe-534732c521a3 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:38.562038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-83424433-9d18-468e-bbe9-a0d199f77d5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '309d94b39e8c403eb5f36cf2ccd9c55d', 'request_id': 'req-83424433-9d18-468e-bbe9-a0d199f77d5d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:38.569229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-83424433-9d18-468e-bbe9-a0d199f77d5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '309d94b39e8c403eb5f36cf2ccd9c55d', 'request_id': 'req-83424433-9d18-468e-bbe9-a0d199f77d5d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:38.590014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d5fb-e012-4d76-b0fe-534732c521a3 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:38.655487 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17432b-a21b-4ed6-a5a8-958d4bb79d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:38.656101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17432b-a21b-4ed6-a5a8-958d4bb79d84 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:38.656376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17432b-a21b-4ed6-a5a8-958d4bb79d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:38.656628 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17432b-a21b-4ed6-a5a8-958d4bb79d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:38.657262 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17432b-a21b-4ed6-a5a8-958d4bb79d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:38.659292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17432b-a21b-4ed6-a5a8-958d4bb79d84 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:38.699080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-83424433-9d18-468e-bbe9-a0d199f77d5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '309d94b39e8c403eb5f36cf2ccd9c55d', 'request_id': 'req-83424433-9d18-468e-bbe9-a0d199f77d5d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:38.714834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-83424433-9d18-468e-bbe9-a0d199f77d5d admin admin] The token cache is being invalidate because project 309d94b39e8c403eb5f36cf2ccd9c55d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:38.715216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-83424433-9d18-468e-bbe9-a0d199f77d5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:38.715963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-83424433-9d18-468e-bbe9-a0d199f77d5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:38.717628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1068/2156] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:38 2022] DELETE /identity/v3/projects/309d94b39e8c403eb5f36cf2ccd9c55d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:38.718948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8125d97-4f33-4376-a9a4-4cf3d8da210a None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:38.719300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8125d97-4f33-4376-a9a4-4cf3d8da210a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:38.719640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8125d97-4f33-4376-a9a4-4cf3d8da210a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:38.752673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-d8125d97-4f33-4376-a9a4-4cf3d8da210a None None] MFA Rules not processed for user `1e77b5549d44484b9be2bb3d7c7dd4df`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:38.753994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d8125d97-4f33-4376-a9a4-4cf3d8da210a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:38.836747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1089/2157] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:38 2022] POST /identity/v3/projects => generated 383 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:38.836747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17432b-a21b-4ed6-a5a8-958d4bb79d84 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:38.838724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ea17432b-a21b-4ed6-a5a8-958d4bb79d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:38.868238 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1069/2158] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:38.869398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8125d97-4f33-4376-a9a4-4cf3d8da210a None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:38.875712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d8125d97-4f33-4376-a9a4-4cf3d8da210a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:38.914245 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8125d97-4f33-4376-a9a4-4cf3d8da210a None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:38.920199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4fe4b-9082-4d2a-9abc-6d382908bf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:38.921198 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4fe4b-9082-4d2a-9abc-6d382908bf0e admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:38.921691 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4fe4b-9082-4d2a-9abc-6d382908bf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:38.922186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4fe4b-9082-4d2a-9abc-6d382908bf0e admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:38.924841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4fe4b-9082-4d2a-9abc-6d382908bf0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:38.925157 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17432b-a21b-4ed6-a5a8-958d4bb79d84 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:38.927632 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4fe4b-9082-4d2a-9abc-6d382908bf0e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:38.932594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:38.933335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:38.933678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:38.934341 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] PATH_INFO: `/v3/users/16eb1dbb721f43e7882219a8ae5dbb03` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:38.935667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16eb1dbb721f43e7882219a8ae5dbb03)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:38.937834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:39.024547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:39.030455 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1070/2159] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:38 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:39.031637 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4fe4b-9082-4d2a-9abc-6d382908bf0e admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:39.037386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0fc4fe4b-9082-4d2a-9abc-6d382908bf0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:39.065313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4fe4b-9082-4d2a-9abc-6d382908bf0e admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:39.077195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb14844-1c84-4418-acdd-a4c297fa090d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:39.078076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb14844-1c84-4418-acdd-a4c297fa090d admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:39.078392 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb14844-1c84-4418-acdd-a4c297fa090d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:39.078655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb14844-1c84-4418-acdd-a4c297fa090d admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:39.079454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb14844-1c84-4418-acdd-a4c297fa090d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:39.081805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb14844-1c84-4418-acdd-a4c297fa090d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:39.111720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16eb1dbb721f43e7882219a8ae5dbb03'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:39.197876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16eb1dbb721f43e7882219a8ae5dbb03'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:39.198584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16eb1dbb721f43e7882219a8ae5dbb03'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:39.199228 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16eb1dbb721f43e7882219a8ae5dbb03'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:39.215917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1071/2160] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:39 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:39.218561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb14844-1c84-4418-acdd-a4c297fa090d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:39.224068 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-feb14844-1c84-4418-acdd-a4c297fa090d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:39.226264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16eb1dbb721f43e7882219a8ae5dbb03'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:39.244738 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:39.249813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1090/2161] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:38 2022] DELETE /identity/v3/users/16eb1dbb721f43e7882219a8ae5dbb03 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:39.250844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:39.253132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:39.354132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f140a-79b9-4973-8173-af396c754ef5 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:39.359016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48693e-cff1-43f9-9dc2-3ead41448364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:39.360318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48693e-cff1-43f9-9dc2-3ead41448364 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:39.360826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48693e-cff1-43f9-9dc2-3ead41448364 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:39.361304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48693e-cff1-43f9-9dc2-3ead41448364 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:39.362188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48693e-cff1-43f9-9dc2-3ead41448364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:39.365054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48693e-cff1-43f9-9dc2-3ead41448364 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:39.429278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb14844-1c84-4418-acdd-a4c297fa090d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:39.436165 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce81e4-21a1-4555-a8ab-9ae78a0c13c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:39.437646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce81e4-21a1-4555-a8ab-9ae78a0c13c3 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:39.437646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce81e4-21a1-4555-a8ab-9ae78a0c13c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:39.437965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce81e4-21a1-4555-a8ab-9ae78a0c13c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:39.438615 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce81e4-21a1-4555-a8ab-9ae78a0c13c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:39.440296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce81e4-21a1-4555-a8ab-9ae78a0c13c3 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:39.463175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1072/2162] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:39.464707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce81e4-21a1-4555-a8ab-9ae78a0c13c3 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:39.476171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1091/2163] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:39 2022] POST /identity/v3/users => generated 294 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:39.480222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48693e-cff1-43f9-9dc2-3ead41448364 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:39.483897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce81e4-21a1-4555-a8ab-9ae78a0c13c3 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:39.488236 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:39.490944 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb864e8e-08c3-4b29-944e-8b3037459ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:39.494778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb864e8e-08c3-4b29-944e-8b3037459ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:39.494778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb864e8e-08c3-4b29-944e-8b3037459ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:39.494778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb864e8e-08c3-4b29-944e-8b3037459ec9 admin admin] PATH_INFO: `/v3/projects/68931eadcdac47c98c71285db2775be5/users/271e3dc8558547949f90e566211019e7/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:39.495034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48693e-cff1-43f9-9dc2-3ead41448364 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:39.505029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd3e1e-b7f0-4b53-9903-4be1eb6ceb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:39.505684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd3e1e-b7f0-4b53-9903-4be1eb6ceb47 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:39.505974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd3e1e-b7f0-4b53-9903-4be1eb6ceb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:39.506247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd3e1e-b7f0-4b53-9903-4be1eb6ceb47 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:39.509505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd3e1e-b7f0-4b53-9903-4be1eb6ceb47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:39.509505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd3e1e-b7f0-4b53-9903-4be1eb6ceb47 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:39.515095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb864e8e-08c3-4b29-944e-8b3037459ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68931eadcdac47c98c71285db2775be5, user_id=271e3dc8558547949f90e566211019e7, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:39.515586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb864e8e-08c3-4b29-944e-8b3037459ec9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:39.523695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1092/2164] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:39.524762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd3e1e-b7f0-4b53-9903-4be1eb6ceb47 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:39.534468 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd3e1e-b7f0-4b53-9903-4be1eb6ceb47 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:39.539349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749d535-2333-4308-91a3-ccc61c451f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:39.539996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749d535-2333-4308-91a3-ccc61c451f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:39.540271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749d535-2333-4308-91a3-ccc61c451f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:39.540558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749d535-2333-4308-91a3-ccc61c451f91 admin admin] PATH_INFO: `/v3/projects/6ca5b51a58cb4c9695c02d235aa1b15b/users/7d41035c863d405b9190727aa8088f35/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:39.555706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749d535-2333-4308-91a3-ccc61c451f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca5b51a58cb4c9695c02d235aa1b15b, user_id=7d41035c863d405b9190727aa8088f35, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:39.556106 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:39.558335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749d535-2333-4308-91a3-ccc61c451f91 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:39.589541 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1073/2165] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:39 2022] PUT /identity/v3/projects/68931eadcdac47c98c71285db2775be5/users/271e3dc8558547949f90e566211019e7/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:39.590663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb864e8e-08c3-4b29-944e-8b3037459ec9 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:39.642290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1093/2166] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:39 2022] PUT /identity/v3/projects/6ca5b51a58cb4c9695c02d235aa1b15b/users/7d41035c863d405b9190727aa8088f35/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:39.643938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749d535-2333-4308-91a3-ccc61c451f91 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:39.665949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb864e8e-08c3-4b29-944e-8b3037459ec9 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:39.747434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335c165-24be-4bc1-a0ee-9804d06a71b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:39.748479 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335c165-24be-4bc1-a0ee-9804d06a71b8 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:39.749519 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335c165-24be-4bc1-a0ee-9804d06a71b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:39.759893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335c165-24be-4bc1-a0ee-9804d06a71b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:39.759893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335c165-24be-4bc1-a0ee-9804d06a71b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:39.759893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335c165-24be-4bc1-a0ee-9804d06a71b8 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:39.760282 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749d535-2333-4308-91a3-ccc61c451f91 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:39.765146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faff1e1-76d9-40fb-8fc3-9cd5562fe41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:39.766547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faff1e1-76d9-40fb-8fc3-9cd5562fe41c admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:39.766904 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faff1e1-76d9-40fb-8fc3-9cd5562fe41c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:39.767250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faff1e1-76d9-40fb-8fc3-9cd5562fe41c admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:39.768096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faff1e1-76d9-40fb-8fc3-9cd5562fe41c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:39.769777 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faff1e1-76d9-40fb-8fc3-9cd5562fe41c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:39.770834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1074/2167] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:39.772077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335c165-24be-4bc1-a0ee-9804d06a71b8 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:39.774132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2335c165-24be-4bc1-a0ee-9804d06a71b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:39.790218 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1094/2168] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:39.790943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faff1e1-76d9-40fb-8fc3-9cd5562fe41c admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:39.792839 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335c165-24be-4bc1-a0ee-9804d06a71b8 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:39.799103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7387e00-483c-4820-a2af-466fe8d3573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:39.800090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7387e00-483c-4820-a2af-466fe8d3573e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:39.801206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7387e00-483c-4820-a2af-466fe8d3573e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:39.801691 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7387e00-483c-4820-a2af-466fe8d3573e admin admin] PATH_INFO: `/v3/users/6d1d1765412b42d8a33d66ad85811d36` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:39.804122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7387e00-483c-4820-a2af-466fe8d3573e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d1d1765412b42d8a33d66ad85811d36)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:39.806878 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7387e00-483c-4820-a2af-466fe8d3573e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:39.806997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faff1e1-76d9-40fb-8fc3-9cd5562fe41c admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:39.817318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686eb56-1a2d-4eec-95eb-f3b3558cf4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:39.818800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686eb56-1a2d-4eec-95eb-f3b3558cf4b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:39.820155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686eb56-1a2d-4eec-95eb-f3b3558cf4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:39.820597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686eb56-1a2d-4eec-95eb-f3b3558cf4b3 admin admin] PATH_INFO: `/v3/projects/68931eadcdac47c98c71285db2775be5/users/271e3dc8558547949f90e566211019e7/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:39.823155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686eb56-1a2d-4eec-95eb-f3b3558cf4b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68931eadcdac47c98c71285db2775be5, user_id=271e3dc8558547949f90e566211019e7, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:39.826852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686eb56-1a2d-4eec-95eb-f3b3558cf4b3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:39.850620 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1095/2169] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:39 2022] PUT /identity/v3/projects/68931eadcdac47c98c71285db2775be5/users/271e3dc8558547949f90e566211019e7/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:39.850620 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686eb56-1a2d-4eec-95eb-f3b3558cf4b3 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:39.935548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686eb56-1a2d-4eec-95eb-f3b3558cf4b3 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:39.944673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3fccb-7c6b-4df8-8606-f6b413942983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:39.944673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3fccb-7c6b-4df8-8606-f6b413942983 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:39.944673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3fccb-7c6b-4df8-8606-f6b413942983 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:39.944673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3fccb-7c6b-4df8-8606-f6b413942983 admin admin] PATH_INFO: `/v3/projects/6ca5b51a58cb4c9695c02d235aa1b15b/users/7d41035c863d405b9190727aa8088f35/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:39.944963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3fccb-7c6b-4df8-8606-f6b413942983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca5b51a58cb4c9695c02d235aa1b15b, user_id=7d41035c863d405b9190727aa8088f35, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:39.947429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3fccb-7c6b-4df8-8606-f6b413942983 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:39.959625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1096/2170] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:39 2022] PUT /identity/v3/projects/6ca5b51a58cb4c9695c02d235aa1b15b/users/7d41035c863d405b9190727aa8088f35/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:39.969518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea876f2c-002c-40d8-b954-1bbd261bd06d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:39.969799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea876f2c-002c-40d8-b954-1bbd261bd06d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:39.970186 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea876f2c-002c-40d8-b954-1bbd261bd06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:40.004974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-ea876f2c-002c-40d8-b954-1bbd261bd06d None None] MFA Rules not processed for user `271e3dc8558547949f90e566211019e7`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:40.006389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ea876f2c-002c-40d8-b954-1bbd261bd06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:40.008945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b7387e00-483c-4820-a2af-466fe8d3573e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d1d1765412b42d8a33d66ad85811d36'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:40.076299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b7387e00-483c-4820-a2af-466fe8d3573e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d1d1765412b42d8a33d66ad85811d36'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:40.077120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b7387e00-483c-4820-a2af-466fe8d3573e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d1d1765412b42d8a33d66ad85811d36'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:40.078166 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b7387e00-483c-4820-a2af-466fe8d3573e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d1d1765412b42d8a33d66ad85811d36'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:40.088918 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1097/2171] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:40.094604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea876f2c-002c-40d8-b954-1bbd261bd06d None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:40.094604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ea876f2c-002c-40d8-b954-1bbd261bd06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:40.096883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b7387e00-483c-4820-a2af-466fe8d3573e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d1d1765412b42d8a33d66ad85811d36'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:40.141031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1075/2172] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:39 2022] DELETE /identity/v3/users/6d1d1765412b42d8a33d66ad85811d36 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:40.141031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f137-da18-4cfd-b72d-88fe4409ebea None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:40.141031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f137-da18-4cfd-b72d-88fe4409ebea None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:40.141031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f137-da18-4cfd-b72d-88fe4409ebea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:40.159469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:40.179775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-0a62f137-da18-4cfd-b72d-88fe4409ebea None None] MFA Rules not processed for user `7d41035c863d405b9190727aa8088f35`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:40.181332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0a62f137-da18-4cfd-b72d-88fe4409ebea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:40.193727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:40.201718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea876f2c-002c-40d8-b954-1bbd261bd06d None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:40.269243 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1076/2173] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:40.271136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f137-da18-4cfd-b72d-88fe4409ebea None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:40.272433 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0a62f137-da18-4cfd-b72d-88fe4409ebea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:40.278149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:40.279468 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:40.280029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:40.280318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] PATH_INFO: `/v3/projects/2ce3a06716b044549ee8f11b99430c72` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:40.281879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce3a06716b044549ee8f11b99430c72)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:40.283671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:40.339217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f137-da18-4cfd-b72d-88fe4409ebea None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:40.339217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5f6e9-eeb1-4456-9611-13af92e79d23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:40.339649 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5f6e9-eeb1-4456-9611-13af92e79d23 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:40.340061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5f6e9-eeb1-4456-9611-13af92e79d23 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:40.340343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5f6e9-eeb1-4456-9611-13af92e79d23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:40.342162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d7a5f6e9-eeb1-4456-9611-13af92e79d23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:40.401841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5f6e9-eeb1-4456-9611-13af92e79d23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:40.403888 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5f6e9-eeb1-4456-9611-13af92e79d23 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:40.413990 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1077/2174] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:40.415301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf10184-d0ab-4465-b824-affef5e632fa None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:40.415587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf10184-d0ab-4465-b824-affef5e632fa None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:40.415855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf10184-d0ab-4465-b824-affef5e632fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:40.440914 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-1bf10184-d0ab-4465-b824-affef5e632fa None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:40.441579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce3a06716b044549ee8f11b99430c72', 'request_id': 'req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:40.442905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-1bf10184-d0ab-4465-b824-affef5e632fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:40.448316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce3a06716b044549ee8f11b99430c72', 'request_id': 'req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:40.510374 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1078/2175] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:40.511270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf10184-d0ab-4465-b824-affef5e632fa None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:40.514143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-1bf10184-d0ab-4465-b824-affef5e632fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:40.544975 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce3a06716b044549ee8f11b99430c72', 'request_id': 'req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:40.566181 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] The token cache is being invalidate because project 2ce3a06716b044549ee8f11b99430c72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:40.566511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:40.568098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:40.576399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1098/2176] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:40 2022] DELETE /identity/v3/projects/2ce3a06716b044549ee8f11b99430c72 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:40.579723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:40.584531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:40.634751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf10184-d0ab-4465-b824-affef5e632fa None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:40.638928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4912e9-9f8c-4e95-bc7f-5584460625d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:40.639758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4912e9-9f8c-4e95-bc7f-5584460625d8 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:40.640083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4912e9-9f8c-4e95-bc7f-5584460625d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:40.641132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4912e9-9f8c-4e95-bc7f-5584460625d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:40.644035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5c4912e9-9f8c-4e95-bc7f-5584460625d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:40.684650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2cf39-1763-4a3e-879b-d1adf9d6d9a0 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:40.695606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c599cd5-86d5-4cc7-923c-5ae24612b074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:40.696632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c599cd5-86d5-4cc7-923c-5ae24612b074 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:40.698098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c599cd5-86d5-4cc7-923c-5ae24612b074 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:40.698591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c599cd5-86d5-4cc7-923c-5ae24612b074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:40.703851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-3c599cd5-86d5-4cc7-923c-5ae24612b074 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:40.723261 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c599cd5-86d5-4cc7-923c-5ae24612b074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:40.725996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c599cd5-86d5-4cc7-923c-5ae24612b074 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:40.742925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1099/2177] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:40.745140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d6b79-cc28-4fe3-96d4-4ead2c56b28e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:40.747983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d6b79-cc28-4fe3-96d4-4ead2c56b28e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:40.747983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d6b79-cc28-4fe3-96d4-4ead2c56b28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:40.748162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4912e9-9f8c-4e95-bc7f-5584460625d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:40.750128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4912e9-9f8c-4e95-bc7f-5584460625d8 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:40.763023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1079/2178] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:40.767343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-301d6b79-cc28-4fe3-96d4-4ead2c56b28e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:40.768613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-301d6b79-cc28-4fe3-96d4-4ead2c56b28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:40.788274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1100/2179] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:40.793006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a1155-452d-4914-b839-1207382ca102 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:40.793300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a1155-452d-4914-b839-1207382ca102 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:40.793574 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a1155-452d-4914-b839-1207382ca102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:40.836667 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-105a1155-452d-4914-b839-1207382ca102 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:40.838321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-105a1155-452d-4914-b839-1207382ca102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:40.853592 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1080/2180] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:40.855689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d6b79-cc28-4fe3-96d4-4ead2c56b28e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:40.869292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d6b79-cc28-4fe3-96d4-4ead2c56b28e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:40.875636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571cf1f-6870-46ee-bc83-66a1350d2ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:40.876251 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571cf1f-6870-46ee-bc83-66a1350d2ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:40.876955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571cf1f-6870-46ee-bc83-66a1350d2ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:40.878996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:40.883953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571cf1f-6870-46ee-bc83-66a1350d2ad1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:40.883953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571cf1f-6870-46ee-bc83-66a1350d2ad1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:40.883953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571cf1f-6870-46ee-bc83-66a1350d2ad1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:40.883953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571cf1f-6870-46ee-bc83-66a1350d2ad1 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:40.906137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1101/2181] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:40.907750 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a1155-452d-4914-b839-1207382ca102 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:40.936790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a1155-452d-4914-b839-1207382ca102 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:40.955485 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56327f6-ed96-476d-a7ef-95c3122519d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:40.956105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56327f6-ed96-476d-a7ef-95c3122519d1 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:40.956386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56327f6-ed96-476d-a7ef-95c3122519d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:40.956655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56327f6-ed96-476d-a7ef-95c3122519d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:40.957435 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56327f6-ed96-476d-a7ef-95c3122519d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:40.959771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56327f6-ed96-476d-a7ef-95c3122519d1 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:41.091952 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1081/2182] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:40 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:57:41.093087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571cf1f-6870-46ee-bc83-66a1350d2ad1 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:41.111258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571cf1f-6870-46ee-bc83-66a1350d2ad1 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:41.174449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d07ed-aaa8-4d90-9964-e4a2e644bafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:41.175438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d07ed-aaa8-4d90-9964-e4a2e644bafd admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:41.176994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d07ed-aaa8-4d90-9964-e4a2e644bafd admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:41.176994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d07ed-aaa8-4d90-9964-e4a2e644bafd admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:41.178639 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d07ed-aaa8-4d90-9964-e4a2e644bafd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:41.182159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d07ed-aaa8-4d90-9964-e4a2e644bafd admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:41.303105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:41.306631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1102/2183] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:41 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:41.311301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56327f6-ed96-476d-a7ef-95c3122519d1 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:41.330294 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56327f6-ed96-476d-a7ef-95c3122519d1 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:41.342115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e9f23-6d7b-4222-8874-b0e007e9955a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:41.344844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e9f23-6d7b-4222-8874-b0e007e9955a admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:41.345131 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e9f23-6d7b-4222-8874-b0e007e9955a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:41.345409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e9f23-6d7b-4222-8874-b0e007e9955a admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:41.347524 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e9f23-6d7b-4222-8874-b0e007e9955a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:41.353187 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e9f23-6d7b-4222-8874-b0e007e9955a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:41.380141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1082/2184] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:41.385550 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d07ed-aaa8-4d90-9964-e4a2e644bafd admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:41.404424 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d07ed-aaa8-4d90-9964-e4a2e644bafd admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:41.420371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5a9cd-f995-4d67-a6fa-256e67a50874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:41.421335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5a9cd-f995-4d67-a6fa-256e67a50874 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:41.421743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5a9cd-f995-4d67-a6fa-256e67a50874 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:41.422112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5a9cd-f995-4d67-a6fa-256e67a50874 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:41.422758 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5a9cd-f995-4d67-a6fa-256e67a50874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:41.424355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5a9cd-f995-4d67-a6fa-256e67a50874 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:41.456652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1103/2185] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:41.464748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e9f23-6d7b-4222-8874-b0e007e9955a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:41.489177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e9f23-6d7b-4222-8874-b0e007e9955a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:41.518477 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c0ff5-eb2b-4f79-919e-27236c242789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:41.521346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c0ff5-eb2b-4f79-919e-27236c242789 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:41.521711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c0ff5-eb2b-4f79-919e-27236c242789 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:41.521997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c0ff5-eb2b-4f79-919e-27236c242789 admin admin] PATH_INFO: `/v3/projects/b5861ff66663413196dea3ec1af30d23/users/cffa286964e14a71a0a38202d05fd74b/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:41.557477 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c0ff5-eb2b-4f79-919e-27236c242789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5861ff66663413196dea3ec1af30d23, user_id=cffa286964e14a71a0a38202d05fd74b, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:41.561182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c0ff5-eb2b-4f79-919e-27236c242789 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:41.568129 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60389b-2029-4159-a2e6-e3f5d4c80003 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:41.568679 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60389b-2029-4159-a2e6-e3f5d4c80003 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:41.569189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60389b-2029-4159-a2e6-e3f5d4c80003 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:41.571648 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1104/2186] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 12:57:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:57:41.588044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75094eee-b3f0-4f26-a513-32104f87b867 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:41.588329 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75094eee-b3f0-4f26-a513-32104f87b867 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:41.588638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75094eee-b3f0-4f26-a513-32104f87b867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:41.610924 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-75094eee-b3f0-4f26-a513-32104f87b867 None None] MFA Rules not processed for user `33e177d86450489cacc86578ba26509d`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:41.612128 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-75094eee-b3f0-4f26-a513-32104f87b867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:41.675386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1105/2187] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 12:57:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:41.686751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:41.692687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75094eee-b3f0-4f26-a513-32104f87b867 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:41.694557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-75094eee-b3f0-4f26-a513-32104f87b867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:41.716858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1083/2188] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:41 2022] PUT /identity/v3/projects/b5861ff66663413196dea3ec1af30d23/users/cffa286964e14a71a0a38202d05fd74b/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:41.716858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c0ff5-eb2b-4f79-919e-27236c242789 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:41.724925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-620c0ff5-eb2b-4f79-919e-27236c242789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:41.733258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75094eee-b3f0-4f26-a513-32104f87b867 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:41.800291 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd61e26-86fb-4da5-830d-c556849381f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:41.800832 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd61e26-86fb-4da5-830d-c556849381f2 service placement] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:41.801790 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd61e26-86fb-4da5-830d-c556849381f2 service placement] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:41.802159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd61e26-86fb-4da5-830d-c556849381f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:41.804341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c0ff5-eb2b-4f79-919e-27236c242789 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:41.810014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:41.810726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:41.811006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:41.811251 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd61e26-86fb-4da5-830d-c556849381f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:41.811373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd admin admin] PATH_INFO: `/v3/projects/0c8f9106e3da46ec8661ed5c920a75bd` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:41.812383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c8f9106e3da46ec8661ed5c920a75bd)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:41.812836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd61e26-86fb-4da5-830d-c556849381f2 service placement] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:41.814263 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:41.825004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1106/2189] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:41.825994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd61e26-86fb-4da5-830d-c556849381f2 service placement] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:41.827417 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fdd61e26-86fb-4da5-830d-c556849381f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:41.844600 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd61e26-86fb-4da5-830d-c556849381f2 service placement] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:41.851355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:41.852000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:41.852345 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:41.852614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] PATH_INFO: `/v3/users/c71eb53fc47c4f82add6fdd3d469230e` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:41.854002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c71eb53fc47c4f82add6fdd3d469230e)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:41.855702 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:41.937939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c8f9106e3da46ec8661ed5c920a75bd', 'request_id': 'req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:41.944427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c8f9106e3da46ec8661ed5c920a75bd', 'request_id': 'req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.039881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c8f9106e3da46ec8661ed5c920a75bd', 'request_id': 'req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.050776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd admin admin] The token cache is being invalidate because project 0c8f9106e3da46ec8661ed5c920a75bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:42.051655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.052406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2ca7a2b7-984b-4685-a9ac-c0364a3436bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.054002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1084/2190] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:41 2022] DELETE /identity/v3/projects/0c8f9106e3da46ec8661ed5c920a75bd => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:42.055219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ef7b2-f4f2-4930-bdec-a3ff8cf7a046 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:42.055501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ef7b2-f4f2-4930-bdec-a3ff8cf7a046 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:42.055773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ef7b2-f4f2-4930-bdec-a3ff8cf7a046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:42.076508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c71eb53fc47c4f82add6fdd3d469230e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.086337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-bb0ef7b2-f4f2-4930-bdec-a3ff8cf7a046 None None] MFA Rules not processed for user `cffa286964e14a71a0a38202d05fd74b`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:42.087567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-bb0ef7b2-f4f2-4930-bdec-a3ff8cf7a046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:42.137667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c71eb53fc47c4f82add6fdd3d469230e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.141258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c71eb53fc47c4f82add6fdd3d469230e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.141258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c71eb53fc47c4f82add6fdd3d469230e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.150956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1085/2191] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:42.151852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ef7b2-f4f2-4930-bdec-a3ff8cf7a046 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:42.152626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c71eb53fc47c4f82add6fdd3d469230e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.153151 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-bb0ef7b2-f4f2-4930-bdec-a3ff8cf7a046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:42.167408 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1107/2192] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:41 2022] DELETE /identity/v3/users/c71eb53fc47c4f82add6fdd3d469230e => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:42.169889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:42.169889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:42.228634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ef7b2-f4f2-4930-bdec-a3ff8cf7a046 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:42.241135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471440d-7fe5-49a5-938d-d03ad19e6bfc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:42.245074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861eb4c-fdea-442b-a6e3-c573f981092a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:42.250536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861eb4c-fdea-442b-a6e3-c573f981092a service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:42.250536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861eb4c-fdea-442b-a6e3-c573f981092a service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:42.250536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861eb4c-fdea-442b-a6e3-c573f981092a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:42.250536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0861eb4c-fdea-442b-a6e3-c573f981092a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:42.310668 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494bb41-0c6f-4f27-bd4e-5ff1d2557e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:42.310668 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494bb41-0c6f-4f27-bd4e-5ff1d2557e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:42.310668 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494bb41-0c6f-4f27-bd4e-5ff1d2557e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:42.310668 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494bb41-0c6f-4f27-bd4e-5ff1d2557e8d admin admin] PATH_INFO: `/v3/users/eea04a5c72aa499f95cec718147fa55f` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:42.311174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494bb41-0c6f-4f27-bd4e-5ff1d2557e8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea04a5c72aa499f95cec718147fa55f)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:42.311174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494bb41-0c6f-4f27-bd4e-5ff1d2557e8d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:42.326457 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861eb4c-fdea-442b-a6e3-c573f981092a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:42.329228 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861eb4c-fdea-442b-a6e3-c573f981092a service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:42.344149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1108/2193] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:42.484447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861eb4c-fdea-442b-a6e3-c573f981092a service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:42.485047 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0861eb4c-fdea-442b-a6e3-c573f981092a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:42.501086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861eb4c-fdea-442b-a6e3-c573f981092a service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:42.517765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e6563-93c5-455a-93a7-1f66a1b1264a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:42.517765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e6563-93c5-455a-93a7-1f66a1b1264a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:42.517765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e6563-93c5-455a-93a7-1f66a1b1264a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:42.517765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e6563-93c5-455a-93a7-1f66a1b1264a admin admin] PATH_INFO: `/v3/projects/9c2a31fedb6843ba8ffea570a374acf5` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:42.518519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e6563-93c5-455a-93a7-1f66a1b1264a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c2a31fedb6843ba8ffea570a374acf5)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:42.518519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e6563-93c5-455a-93a7-1f66a1b1264a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:42.577522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e494bb41-0c6f-4f27-bd4e-5ff1d2557e8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea04a5c72aa499f95cec718147fa55f'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.635255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-fa1e6563-93c5-455a-93a7-1f66a1b1264a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c2a31fedb6843ba8ffea570a374acf5', 'request_id': 'req-fa1e6563-93c5-455a-93a7-1f66a1b1264a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.641290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-fa1e6563-93c5-455a-93a7-1f66a1b1264a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c2a31fedb6843ba8ffea570a374acf5', 'request_id': 'req-fa1e6563-93c5-455a-93a7-1f66a1b1264a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.647298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e494bb41-0c6f-4f27-bd4e-5ff1d2557e8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea04a5c72aa499f95cec718147fa55f'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.647569 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:42.647932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e494bb41-0c6f-4f27-bd4e-5ff1d2557e8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea04a5c72aa499f95cec718147fa55f'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.648544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e494bb41-0c6f-4f27-bd4e-5ff1d2557e8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea04a5c72aa499f95cec718147fa55f'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.656161 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e494bb41-0c6f-4f27-bd4e-5ff1d2557e8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea04a5c72aa499f95cec718147fa55f'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.674574 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1086/2194] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:42 2022] DELETE /identity/v3/users/eea04a5c72aa499f95cec718147fa55f => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:42.716382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:42.739370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-fa1e6563-93c5-455a-93a7-1f66a1b1264a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c2a31fedb6843ba8ffea570a374acf5', 'request_id': 'req-fa1e6563-93c5-455a-93a7-1f66a1b1264a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.777989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-fa1e6563-93c5-455a-93a7-1f66a1b1264a admin admin] The token cache is being invalidate because project 9c2a31fedb6843ba8ffea570a374acf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:42.778294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-fa1e6563-93c5-455a-93a7-1f66a1b1264a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.779001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-fa1e6563-93c5-455a-93a7-1f66a1b1264a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:42.787332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1109/2195] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:42 2022] DELETE /identity/v3/projects/9c2a31fedb6843ba8ffea570a374acf5 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:42.803215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93375a95-78b1-4e51-9083-50d732034b30 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:42.803215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93375a95-78b1-4e51-9083-50d732034b30 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:42.803215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93375a95-78b1-4e51-9083-50d732034b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:42.823671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-93375a95-78b1-4e51-9083-50d732034b30 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:42.823671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-93375a95-78b1-4e51-9083-50d732034b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:42.906416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1087/2196] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:42.914306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6c5e6-3b94-40a4-a3c5-2d014244ebec None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:42.914689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6c5e6-3b94-40a4-a3c5-2d014244ebec None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:42.915020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6c5e6-3b94-40a4-a3c5-2d014244ebec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:42.954781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-2cb6c5e6-3b94-40a4-a3c5-2d014244ebec None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:42.954781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2cb6c5e6-3b94-40a4-a3c5-2d014244ebec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:42.982519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1110/2197] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:42.985804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93375a95-78b1-4e51-9083-50d732034b30 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:42.998994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93375a95-78b1-4e51-9083-50d732034b30 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:43.005376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b5123-5a1f-4594-a2ae-00dbd58955f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:43.006042 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b5123-5a1f-4594-a2ae-00dbd58955f9 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.006347 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b5123-5a1f-4594-a2ae-00dbd58955f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.006664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b5123-5a1f-4594-a2ae-00dbd58955f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.007311 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b5123-5a1f-4594-a2ae-00dbd58955f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:43.007609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b5123-5a1f-4594-a2ae-00dbd58955f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:43.009372 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b5123-5a1f-4594-a2ae-00dbd58955f9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:43.027004 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1088/2198] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:43.029020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6c5e6-3b94-40a4-a3c5-2d014244ebec None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:43.041113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6c5e6-3b94-40a4-a3c5-2d014244ebec None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:43.047264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cd3c6-fb18-4f2c-b733-81b71dab996b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:43.047874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cd3c6-fb18-4f2c-b733-81b71dab996b admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.048308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cd3c6-fb18-4f2c-b733-81b71dab996b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.048586 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cd3c6-fb18-4f2c-b733-81b71dab996b admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.050033 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cd3c6-fb18-4f2c-b733-81b71dab996b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:43.052632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cd3c6-fb18-4f2c-b733-81b71dab996b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:43.145717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1111/2199] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:43 2022] POST /identity/v3/projects => generated 391 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:57:43.149764 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b5123-5a1f-4594-a2ae-00dbd58955f9 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:43.168154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b5123-5a1f-4594-a2ae-00dbd58955f9 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:43.195281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cd3c6-fb18-4f2c-b733-81b71dab996b admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:43.197418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a18cd3c6-fb18-4f2c-b733-81b71dab996b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:43.245260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172b7fd-7be5-473c-8907-280d8bbc478d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:43.246259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172b7fd-7be5-473c-8907-280d8bbc478d admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.246591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172b7fd-7be5-473c-8907-280d8bbc478d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.250529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172b7fd-7be5-473c-8907-280d8bbc478d admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.250529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172b7fd-7be5-473c-8907-280d8bbc478d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:43.250724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:43.250826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172b7fd-7be5-473c-8907-280d8bbc478d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:43.276243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cd3c6-fb18-4f2c-b733-81b71dab996b admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:43.280017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e60335-3e9f-4e37-a875-9954058cf31c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:43.280650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e60335-3e9f-4e37-a875-9954058cf31c service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.280925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e60335-3e9f-4e37-a875-9954058cf31c service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.281195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e60335-3e9f-4e37-a875-9954058cf31c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.282965 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b6e60335-3e9f-4e37-a875-9954058cf31c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:43.340590 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e60335-3e9f-4e37-a875-9954058cf31c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:43.342036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e60335-3e9f-4e37-a875-9954058cf31c service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:43.351552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1112/2200] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:57:43.352377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e60335-3e9f-4e37-a875-9954058cf31c service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:43.353679 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b6e60335-3e9f-4e37-a875-9954058cf31c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:43.371234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e60335-3e9f-4e37-a875-9954058cf31c service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:43.375531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad5593-3e94-402f-ad74-784af689fd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:43.376384 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad5593-3e94-402f-ad74-784af689fd46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.376857 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad5593-3e94-402f-ad74-784af689fd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.377307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad5593-3e94-402f-ad74-784af689fd46 admin admin] PATH_INFO: `/v3/projects/35f3224115fb46ac956b15f733fdacec` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.378674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad5593-3e94-402f-ad74-784af689fd46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35f3224115fb46ac956b15f733fdacec)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:43.381975 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad5593-3e94-402f-ad74-784af689fd46 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:43.400367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1089/2201] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:43 2022] POST /identity/v3/users => generated 296 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:43.414940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172b7fd-7be5-473c-8907-280d8bbc478d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:43.417859 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172b7fd-7be5-473c-8907-280d8bbc478d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:43.423758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2e37b-3e3d-4a2e-9f27-0068c4770048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:43.424370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2e37b-3e3d-4a2e-9f27-0068c4770048 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.424653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2e37b-3e3d-4a2e-9f27-0068c4770048 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.425490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2e37b-3e3d-4a2e-9f27-0068c4770048 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.426048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2e37b-3e3d-4a2e-9f27-0068c4770048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:43.428444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2e37b-3e3d-4a2e-9f27-0068c4770048 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:43.440044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1090/2202] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:43.444931 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2e37b-3e3d-4a2e-9f27-0068c4770048 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:43.455345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2e37b-3e3d-4a2e-9f27-0068c4770048 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:43.461627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdc0c9-04e0-4a54-abc5-1631b9475574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:43.469108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdc0c9-04e0-4a54-abc5-1631b9475574 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.469108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdc0c9-04e0-4a54-abc5-1631b9475574 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.469108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdc0c9-04e0-4a54-abc5-1631b9475574 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.469108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bdc0c9-04e0-4a54-abc5-1631b9475574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:43.469108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bdc0c9-04e0-4a54-abc5-1631b9475574 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:43.475381 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1091/2203] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:43.479444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdc0c9-04e0-4a54-abc5-1631b9475574 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:43.489248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdc0c9-04e0-4a54-abc5-1631b9475574 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:43.493887 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ede01-9892-462f-a397-761d5812f4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:43.494541 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ede01-9892-462f-a397-761d5812f4ef admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.494830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ede01-9892-462f-a397-761d5812f4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.495108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ede01-9892-462f-a397-761d5812f4ef admin admin] PATH_INFO: `/v3/projects/bf202f6eebd442e988e0f7307950a6c8/users/48f14caeb8f944c6aaf27913972869c7/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.515028 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ede01-9892-462f-a397-761d5812f4ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf202f6eebd442e988e0f7307950a6c8, user_id=48f14caeb8f944c6aaf27913972869c7, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:43.515028 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ede01-9892-462f-a397-761d5812f4ef admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:43.528853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-77ad5593-3e94-402f-ad74-784af689fd46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35f3224115fb46ac956b15f733fdacec', 'request_id': 'req-77ad5593-3e94-402f-ad74-784af689fd46'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:43.535355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-77ad5593-3e94-402f-ad74-784af689fd46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35f3224115fb46ac956b15f733fdacec', 'request_id': 'req-77ad5593-3e94-402f-ad74-784af689fd46'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:43.607496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1092/2204] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:43 2022] PUT /identity/v3/projects/bf202f6eebd442e988e0f7307950a6c8/users/48f14caeb8f944c6aaf27913972869c7/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:43.612306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d54bc9-a61c-4def-815e-94307dbafe2e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.612592 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d54bc9-a61c-4def-815e-94307dbafe2e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.612867 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d54bc9-a61c-4def-815e-94307dbafe2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.633924 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-77ad5593-3e94-402f-ad74-784af689fd46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35f3224115fb46ac956b15f733fdacec', 'request_id': 'req-77ad5593-3e94-402f-ad74-784af689fd46'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:43.634056 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-75d54bc9-a61c-4def-815e-94307dbafe2e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:43.634056 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-75d54bc9-a61c-4def-815e-94307dbafe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:43.636753 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-77ad5593-3e94-402f-ad74-784af689fd46 admin admin] The token cache is being invalidate because project 35f3224115fb46ac956b15f733fdacec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:43.637053 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-77ad5593-3e94-402f-ad74-784af689fd46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:43.637712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-77ad5593-3e94-402f-ad74-784af689fd46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:43.639260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1113/2205] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:43 2022] DELETE /identity/v3/projects/35f3224115fb46ac956b15f733fdacec => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:43.645904 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5d2b7-8042-4653-a1de-34a1a474abdc None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.646622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5d2b7-8042-4653-a1de-34a1a474abdc None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.647431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5d2b7-8042-4653-a1de-34a1a474abdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.668645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-05b5d2b7-8042-4653-a1de-34a1a474abdc None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:43.669856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-05b5d2b7-8042-4653-a1de-34a1a474abdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:43.694588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1093/2206] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:43.704484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d54bc9-a61c-4def-815e-94307dbafe2e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:43.705919 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-75d54bc9-a61c-4def-815e-94307dbafe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:43.730790 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1114/2207] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:43.739033 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09bb2c-4391-496e-9db8-4c6b4aa3990b None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.739369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09bb2c-4391-496e-9db8-4c6b4aa3990b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.739656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09bb2c-4391-496e-9db8-4c6b4aa3990b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.761359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-6c09bb2c-4391-496e-9db8-4c6b4aa3990b None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:43.762923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6c09bb2c-4391-496e-9db8-4c6b4aa3990b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:43.765762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d54bc9-a61c-4def-815e-94307dbafe2e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:43.770500 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf1546-2747-4b50-9881-02bbc7e40b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:43.771129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf1546-2747-4b50-9881-02bbc7e40b0c service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.771410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf1546-2747-4b50-9881-02bbc7e40b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.771945 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf1546-2747-4b50-9881-02bbc7e40b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.772061 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1115/2208] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:43.776059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09bb2c-4391-496e-9db8-4c6b4aa3990b None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:43.781687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf1546-2747-4b50-9881-02bbc7e40b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:43.783152 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf1546-2747-4b50-9881-02bbc7e40b0c service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:43.785168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09bb2c-4391-496e-9db8-4c6b4aa3990b None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:43.789997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc8468-25ff-473b-95aa-093433b8c90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:43.790618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc8468-25ff-473b-95aa-093433b8c90e admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.790900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc8468-25ff-473b-95aa-093433b8c90e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.791173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc8468-25ff-473b-95aa-093433b8c90e admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.791818 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc8468-25ff-473b-95aa-093433b8c90e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:43.792119 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc8468-25ff-473b-95aa-093433b8c90e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:43.792752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1094/2209] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:57:43.793782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc8468-25ff-473b-95aa-093433b8c90e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:43.800254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:43.806511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1116/2210] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:57:43.810795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf1546-2747-4b50-9881-02bbc7e40b0c service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:43.820902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf1546-2747-4b50-9881-02bbc7e40b0c service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:43.825594 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340d67c-2763-4e46-aad6-5118345967d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:43.826190 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340d67c-2763-4e46-aad6-5118345967d4 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.826468 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340d67c-2763-4e46-aad6-5118345967d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.826825 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340d67c-2763-4e46-aad6-5118345967d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.827508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340d67c-2763-4e46-aad6-5118345967d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:43.829671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340d67c-2763-4e46-aad6-5118345967d4 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:43.918240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1095/2211] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:43 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:57:43.922319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc8468-25ff-473b-95aa-093433b8c90e admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:43.939324 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc8468-25ff-473b-95aa-093433b8c90e admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:43.997194 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97fcf1-1871-4a03-badc-e420d913a8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:43.997804 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97fcf1-1871-4a03-badc-e420d913a8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:43.998773 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97fcf1-1871-4a03-badc-e420d913a8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:43.998773 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97fcf1-1871-4a03-badc-e420d913a8d9 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:43.999062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97fcf1-1871-4a03-badc-e420d913a8d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:44.001269 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97fcf1-1871-4a03-badc-e420d913a8d9 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:44.121062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1117/2212] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:43 2022] POST /identity/v3/users => generated 284 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:44.127522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340d67c-2763-4e46-aad6-5118345967d4 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:44.153206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340d67c-2763-4e46-aad6-5118345967d4 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:44.158843 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420d560-43fb-4468-b9ab-c7f687cde27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:44.159462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420d560-43fb-4468-b9ab-c7f687cde27f admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:44.159739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420d560-43fb-4468-b9ab-c7f687cde27f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:44.160011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420d560-43fb-4468-b9ab-c7f687cde27f admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:44.161334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420d560-43fb-4468-b9ab-c7f687cde27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:44.162981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420d560-43fb-4468-b9ab-c7f687cde27f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:44.173525 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1096/2213] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:44.178074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97fcf1-1871-4a03-badc-e420d913a8d9 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:44.188152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97fcf1-1871-4a03-badc-e420d913a8d9 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:44.193241 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3040e81-2f08-49f1-b290-89b1013b996c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:44.193871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3040e81-2f08-49f1-b290-89b1013b996c admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:44.194147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3040e81-2f08-49f1-b290-89b1013b996c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:44.194418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3040e81-2f08-49f1-b290-89b1013b996c admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:44.195100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3040e81-2f08-49f1-b290-89b1013b996c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:44.196672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3040e81-2f08-49f1-b290-89b1013b996c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:44.207928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1118/2214] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:57:44.212283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420d560-43fb-4468-b9ab-c7f687cde27f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:44.221944 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420d560-43fb-4468-b9ab-c7f687cde27f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:44.227687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65ba97-7889-4f11-9e04-abe3cf3ef984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:44.228321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65ba97-7889-4f11-9e04-abe3cf3ef984 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:44.232279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65ba97-7889-4f11-9e04-abe3cf3ef984 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:44.232279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65ba97-7889-4f11-9e04-abe3cf3ef984 admin admin] PATH_INFO: `/v3/projects/deb5ae2fa5db48949e56181fbc70f09a/users/2879dddbb34842feb8f01d98fe9c0ec9/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:44.244073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65ba97-7889-4f11-9e04-abe3cf3ef984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deb5ae2fa5db48949e56181fbc70f09a, user_id=2879dddbb34842feb8f01d98fe9c0ec9, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:44.246720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65ba97-7889-4f11-9e04-abe3cf3ef984 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:44.368129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1097/2215] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:44 2022] PUT /identity/v3/projects/deb5ae2fa5db48949e56181fbc70f09a/users/2879dddbb34842feb8f01d98fe9c0ec9/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:44.378333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a17ad1-50d4-48b8-bf4e-111f14be8ad9 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:44.378647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a17ad1-50d4-48b8-bf4e-111f14be8ad9 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:44.378944 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a17ad1-50d4-48b8-bf4e-111f14be8ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:44.410951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b8a17ad1-50d4-48b8-bf4e-111f14be8ad9 None None] MFA Rules not processed for user `2879dddbb34842feb8f01d98fe9c0ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:44.411985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b8a17ad1-50d4-48b8-bf4e-111f14be8ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:44.482553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1119/2216] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:44.499790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65ba97-7889-4f11-9e04-abe3cf3ef984 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:44.501562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7f65ba97-7889-4f11-9e04-abe3cf3ef984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:44.559236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65ba97-7889-4f11-9e04-abe3cf3ef984 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:44.563208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76611069-2d1b-4b20-96b5-aa1cfd1064d5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff8fcb7fa2740bbb801badf800d2d6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:44.564052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76611069-2d1b-4b20-96b5-aa1cfd1064d5 service glance] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:44.564267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76611069-2d1b-4b20-96b5-aa1cfd1064d5 service glance] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:44.564541 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76611069-2d1b-4b20-96b5-aa1cfd1064d5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:44.574040 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76611069-2d1b-4b20-96b5-aa1cfd1064d5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:44.575552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76611069-2d1b-4b20-96b5-aa1cfd1064d5 service glance] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:44.585474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1098/2217] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:57:44.603730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:44.870287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:45.561997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:46.372001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:46.729616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a17ad1-50d4-48b8-bf4e-111f14be8ad9 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:46.731165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b8a17ad1-50d4-48b8-bf4e-111f14be8ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:46.834112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a17ad1-50d4-48b8-bf4e-111f14be8ad9 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:46.839399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24434a77-8ca1-4aa6-98d6-20dcceb81026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:46.844145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24434a77-8ca1-4aa6-98d6-20dcceb81026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:46.844145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:46.844145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24434a77-8ca1-4aa6-98d6-20dcceb81026 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:46.844145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24434a77-8ca1-4aa6-98d6-20dcceb81026 admin admin] PATH_INFO: `/v3/users/1e77b5549d44484b9be2bb3d7c7dd4df` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:46.847154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24434a77-8ca1-4aa6-98d6-20dcceb81026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e77b5549d44484b9be2bb3d7c7dd4df)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:46.853917 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24434a77-8ca1-4aa6-98d6-20dcceb81026 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:47.185735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76611069-2d1b-4b20-96b5-aa1cfd1064d5 service glance] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:47.188159 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-76611069-2d1b-4b20-96b5-aa1cfd1064d5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:47.193433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-24434a77-8ca1-4aa6-98d6-20dcceb81026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e77b5549d44484b9be2bb3d7c7dd4df'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:47.303731 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-24434a77-8ca1-4aa6-98d6-20dcceb81026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e77b5549d44484b9be2bb3d7c7dd4df'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:47.304847 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-24434a77-8ca1-4aa6-98d6-20dcceb81026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e77b5549d44484b9be2bb3d7c7dd4df'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:47.306610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-24434a77-8ca1-4aa6-98d6-20dcceb81026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e77b5549d44484b9be2bb3d7c7dd4df'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:47.312082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76611069-2d1b-4b20-96b5-aa1cfd1064d5 service glance] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:47.316305 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba4857-f92a-4476-8524-05c8e5e05785 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:47.317034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4857-f92a-4476-8524-05c8e5e05785 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:47.317313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4857-f92a-4476-8524-05c8e5e05785 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:47.317740 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4857-f92a-4476-8524-05c8e5e05785 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:47.319805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-57ba4857-f92a-4476-8524-05c8e5e05785 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:47.324835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-24434a77-8ca1-4aa6-98d6-20dcceb81026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e77b5549d44484b9be2bb3d7c7dd4df'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:47.370823 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1120/2218] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:46 2022] DELETE /identity/v3/users/1e77b5549d44484b9be2bb3d7c7dd4df => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:47.386967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b8adc-ebe8-426e-9e12-a6257fd0a5b0 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:47.386967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b8adc-ebe8-426e-9e12-a6257fd0a5b0 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:47.386967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b8adc-ebe8-426e-9e12-a6257fd0a5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:47.421816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-122b8adc-ebe8-426e-9e12-a6257fd0a5b0 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:47.423127 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-122b8adc-ebe8-426e-9e12-a6257fd0a5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:47.462961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba4857-f92a-4476-8524-05c8e5e05785 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:47.462961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba4857-f92a-4476-8524-05c8e5e05785 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:47.480002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1099/2219] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:47.505877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1121/2220] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:47.524760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba4857-f92a-4476-8524-05c8e5e05785 service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:47.526959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-57ba4857-f92a-4476-8524-05c8e5e05785 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:47.606311 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba4857-f92a-4476-8524-05c8e5e05785 service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:47.681699 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e490f-86f5-4d6d-97f3-fb7a8f99dc4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:47.682299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e490f-86f5-4d6d-97f3-fb7a8f99dc4f service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:47.682618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e490f-86f5-4d6d-97f3-fb7a8f99dc4f service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:47.682960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e490f-86f5-4d6d-97f3-fb7a8f99dc4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:47.695574 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e490f-86f5-4d6d-97f3-fb7a8f99dc4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:47.700937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e490f-86f5-4d6d-97f3-fb7a8f99dc4f service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:47.708844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1100/2221] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:47.916795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:48.356981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b8adc-ebe8-426e-9e12-a6257fd0a5b0 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:48.358943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-122b8adc-ebe8-426e-9e12-a6257fd0a5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:48.372857 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b8adc-ebe8-426e-9e12-a6257fd0a5b0 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:48.377909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:48.389636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:48.390624 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:48.391099 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:48.392333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] PATH_INFO: `/v3/projects/b049acdc82024f5480b9804239437d85` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:48.394231 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b049acdc82024f5480b9804239437d85)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:48.403277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:48.547316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b049acdc82024f5480b9804239437d85', 'request_id': 'req-9c568f8b-4495-4a13-ae23-c387eedbc1fc'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:48.554535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b049acdc82024f5480b9804239437d85', 'request_id': 'req-9c568f8b-4495-4a13-ae23-c387eedbc1fc'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:48.638939 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b049acdc82024f5480b9804239437d85', 'request_id': 'req-9c568f8b-4495-4a13-ae23-c387eedbc1fc'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:48.651728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] The token cache is being invalidate because project b049acdc82024f5480b9804239437d85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:48.652021 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:48.652684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:48.654195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1122/2222] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:48 2022] DELETE /identity/v3/projects/b049acdc82024f5480b9804239437d85 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:48.658422 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e490f-86f5-4d6d-97f3-fb7a8f99dc4f service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:48.668816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e490f-86f5-4d6d-97f3-fb7a8f99dc4f service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:48.737293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc97ae1-dccb-46e7-9fbf-1f4efc80b5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:48.738002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc97ae1-dccb-46e7-9fbf-1f4efc80b5ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:48.738495 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc97ae1-dccb-46e7-9fbf-1f4efc80b5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:48.739110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc97ae1-dccb-46e7-9fbf-1f4efc80b5ee admin admin] PATH_INFO: `/v3/users/7d41035c863d405b9190727aa8088f35` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:48.740602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc97ae1-dccb-46e7-9fbf-1f4efc80b5ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d41035c863d405b9190727aa8088f35)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:48.745073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc97ae1-dccb-46e7-9fbf-1f4efc80b5ee admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:48.974990 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3dc97ae1-dccb-46e7-9fbf-1f4efc80b5ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d41035c863d405b9190727aa8088f35'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:49.073959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3dc97ae1-dccb-46e7-9fbf-1f4efc80b5ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d41035c863d405b9190727aa8088f35'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:49.073959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3dc97ae1-dccb-46e7-9fbf-1f4efc80b5ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d41035c863d405b9190727aa8088f35'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:49.073959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3dc97ae1-dccb-46e7-9fbf-1f4efc80b5ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d41035c863d405b9190727aa8088f35'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:49.087663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3dc97ae1-dccb-46e7-9fbf-1f4efc80b5ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d41035c863d405b9190727aa8088f35'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:49.113439 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1101/2223] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:48 2022] DELETE /identity/v3/users/7d41035c863d405b9190727aa8088f35 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:49.465032 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:49.607602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:49.609138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:49.639801 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c568f8b-4495-4a13-ae23-c387eedbc1fc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:49.751770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:49.755936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:49.755936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:49.755936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] PATH_INFO: `/v3/projects/6ca5b51a58cb4c9695c02d235aa1b15b` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:49.755936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca5b51a58cb4c9695c02d235aa1b15b)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:49.757597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:49.905723 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35e3b2-70ad-455b-b4e9-0fa4218fc1a4 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:49.906232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35e3b2-70ad-455b-b4e9-0fa4218fc1a4 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:49.906318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35e3b2-70ad-455b-b4e9-0fa4218fc1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:49.937470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca5b51a58cb4c9695c02d235aa1b15b', 'request_id': 'req-409d7efe-be1e-4e96-b401-7f050882016a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:49.950243 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-7d35e3b2-70ad-455b-b4e9-0fa4218fc1a4 None None] MFA Rules not processed for user `48f14caeb8f944c6aaf27913972869c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:49.956370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7d35e3b2-70ad-455b-b4e9-0fa4218fc1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:49.956492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca5b51a58cb4c9695c02d235aa1b15b', 'request_id': 'req-409d7efe-be1e-4e96-b401-7f050882016a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:50.024101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca5b51a58cb4c9695c02d235aa1b15b', 'request_id': 'req-409d7efe-be1e-4e96-b401-7f050882016a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:50.036081 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1102/2224] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:50.036451 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] The token cache is being invalidate because project 6ca5b51a58cb4c9695c02d235aa1b15b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:50.036822 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:50.037489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:50.039018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1123/2225] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:49 2022] DELETE /identity/v3/projects/6ca5b51a58cb4c9695c02d235aa1b15b => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:50.048209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aefd6f-b8c5-470c-9f4c-2028d2264867 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:50.048505 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aefd6f-b8c5-470c-9f4c-2028d2264867 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:50.048920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aefd6f-b8c5-470c-9f4c-2028d2264867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:50.078118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-63aefd6f-b8c5-470c-9f4c-2028d2264867 None None] MFA Rules not processed for user `48f14caeb8f944c6aaf27913972869c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:50.079680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-63aefd6f-b8c5-470c-9f4c-2028d2264867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:50.087731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1103/2226] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:50.093812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:50.103242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:50.105801 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:50.218895 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7efe-be1e-4e96-b401-7f050882016a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:50.225530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b77286-0c70-4e25-8e88-6371819d75f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:50.226165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b77286-0c70-4e25-8e88-6371819d75f3 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:50.226700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b77286-0c70-4e25-8e88-6371819d75f3 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:50.226928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b77286-0c70-4e25-8e88-6371819d75f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:50.228841 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f5b77286-0c70-4e25-8e88-6371819d75f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:50.241097 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b77286-0c70-4e25-8e88-6371819d75f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:50.242924 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b77286-0c70-4e25-8e88-6371819d75f3 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:50.259110 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1124/2227] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:50.304471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:50.784849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:50.812672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:51.022282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c9d2-ff66-4ad0-a609-3f804042bc59 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:51.022835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c9d2-ff66-4ad0-a609-3f804042bc59 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:51.022960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c9d2-ff66-4ad0-a609-3f804042bc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:51.028403 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:51.056298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-e674c9d2-ff66-4ad0-a609-3f804042bc59 None None] MFA Rules not processed for user `48f14caeb8f944c6aaf27913972869c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:51.057788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e674c9d2-ff66-4ad0-a609-3f804042bc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:51.074275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1104/2228] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:51.488048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:51.574374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:51.799984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:52.116614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:52.590868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:52.772171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:53.687754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:53.912100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:54.361153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:54.581359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:55.503833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:55.828468 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b77286-0c70-4e25-8e88-6371819d75f3 service nova] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:55.830993 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f5b77286-0c70-4e25-8e88-6371819d75f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:55.902431 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b77286-0c70-4e25-8e88-6371819d75f3 service nova] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:55.908018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9d547-93cf-4829-afcc-7aa85a5e12b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:55.908652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9d547-93cf-4829-afcc-7aa85a5e12b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:55.908935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9d547-93cf-4829-afcc-7aa85a5e12b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:55.909213 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9d547-93cf-4829-afcc-7aa85a5e12b1 admin admin] PATH_INFO: `/v3/users/7817e876611b42498db2fa5304c89a75` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:55.911240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9d547-93cf-4829-afcc-7aa85a5e12b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7817e876611b42498db2fa5304c89a75)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:55.913158 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9d547-93cf-4829-afcc-7aa85a5e12b1 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:56.049432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c9d2-ff66-4ad0-a609-3f804042bc59 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:56.051605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e674c9d2-ff66-4ad0-a609-3f804042bc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:56.069562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c9d2-ff66-4ad0-a609-3f804042bc59 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:56.074267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:56.074941 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:56.075219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:56.075489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] PATH_INFO: `/v3/users/cffa286964e14a71a0a38202d05fd74b` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:56.076598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cffa286964e14a71a0a38202d05fd74b)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:56.078897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:56.130423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-13a9d547-93cf-4829-afcc-7aa85a5e12b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7817e876611b42498db2fa5304c89a75'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:56.213092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-13a9d547-93cf-4829-afcc-7aa85a5e12b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7817e876611b42498db2fa5304c89a75'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:56.213651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:56.213977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-13a9d547-93cf-4829-afcc-7aa85a5e12b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7817e876611b42498db2fa5304c89a75'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:56.214730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-13a9d547-93cf-4829-afcc-7aa85a5e12b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7817e876611b42498db2fa5304c89a75'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:56.230304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-13a9d547-93cf-4829-afcc-7aa85a5e12b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7817e876611b42498db2fa5304c89a75'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:56.244178 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1125/2229] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:55 2022] DELETE /identity/v3/users/7817e876611b42498db2fa5304c89a75 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:56.248743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95772d1d-4d47-4623-91ca-684c1f687398 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:56.249087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95772d1d-4d47-4623-91ca-684c1f687398 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:56.249483 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95772d1d-4d47-4623-91ca-684c1f687398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:56.271106 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-95772d1d-4d47-4623-91ca-684c1f687398 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:56.273985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-95772d1d-4d47-4623-91ca-684c1f687398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:56.288370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cffa286964e14a71a0a38202d05fd74b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:56.333645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1126/2230] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:56.344072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95772d1d-4d47-4623-91ca-684c1f687398 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:56.345577 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-95772d1d-4d47-4623-91ca-684c1f687398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:56.448742 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95772d1d-4d47-4623-91ca-684c1f687398 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:56.453682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0df63-6a1e-42ab-af67-b9e72563d7f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:56.453682 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0df63-6a1e-42ab-af67-b9e72563d7f1 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:56.453969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0df63-6a1e-42ab-af67-b9e72563d7f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:56.454212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0df63-6a1e-42ab-af67-b9e72563d7f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:56.456086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cffa286964e14a71a0a38202d05fd74b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:56.456699 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cffa286964e14a71a0a38202d05fd74b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:56.458044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cffa286964e14a71a0a38202d05fd74b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:56.472208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0df63-6a1e-42ab-af67-b9e72563d7f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:56.475338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0df63-6a1e-42ab-af67-b9e72563d7f1 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:56.477837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cffa286964e14a71a0a38202d05fd74b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:56.504069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1127/2231] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:56.510364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1105/2232] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:56 2022] DELETE /identity/v3/users/cffa286964e14a71a0a38202d05fd74b => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:56.515045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3eaf64-67a4-4f09-bd18-e5b066b32d87 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:56.515648 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3eaf64-67a4-4f09-bd18-e5b066b32d87 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:56.516132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3eaf64-67a4-4f09-bd18-e5b066b32d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:56.541180 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-4e3eaf64-67a4-4f09-bd18-e5b066b32d87 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:56.542629 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4e3eaf64-67a4-4f09-bd18-e5b066b32d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:56.631150 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1128/2233] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:56.631898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:56.633384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:56.702582 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4cc5b-b6de-4736-81ab-a26edab0ff5f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:56.706790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f00ac-613e-4999-9124-88d4ae16944d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:56.708510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f00ac-613e-4999-9124-88d4ae16944d service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:56.709203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f00ac-613e-4999-9124-88d4ae16944d service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:56.709647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f00ac-613e-4999-9124-88d4ae16944d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:56.722286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f00ac-613e-4999-9124-88d4ae16944d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:56.723883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f00ac-613e-4999-9124-88d4ae16944d service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:56.734400 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1106/2234] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:57:57.041303 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:57.049296 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3eaf64-67a4-4f09-bd18-e5b066b32d87 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:57.051731 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4e3eaf64-67a4-4f09-bd18-e5b066b32d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:57.067974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3eaf64-67a4-4f09-bd18-e5b066b32d87 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:57.080491 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:57.081679 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:57.082065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:57.083141 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] PATH_INFO: `/v3/projects/d1809431125f4ca3bbcd4e84992e340e` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:57.085586 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1809431125f4ca3bbcd4e84992e340e)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:57.089776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:57.116940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:57.219355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1809431125f4ca3bbcd4e84992e340e', 'request_id': 'req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.225838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1809431125f4ca3bbcd4e84992e340e', 'request_id': 'req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.323496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1809431125f4ca3bbcd4e84992e340e', 'request_id': 'req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.331921 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f00ac-613e-4999-9124-88d4ae16944d service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:57.335503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0c9f00ac-613e-4999-9124-88d4ae16944d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:57.342577 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] The token cache is being invalidate because project d1809431125f4ca3bbcd4e84992e340e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:57.342871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.343686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.346817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1129/2235] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:57 2022] DELETE /identity/v3/projects/d1809431125f4ca3bbcd4e84992e340e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:57.355283 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:57.356673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:57.372752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f00ac-613e-4999-9124-88d4ae16944d service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:57.430092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6dfe5-d4c0-4eb6-b192-33614740b1c3 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:57.434805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3e3f2-dfd7-445d-9e8a-300e7583b356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:57.435464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3e3f2-dfd7-445d-9e8a-300e7583b356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:57.435742 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3e3f2-dfd7-445d-9e8a-300e7583b356 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:57.435838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ecf46-cb7e-45d8-9af2-310dea23d354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:57.436016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3e3f2-dfd7-445d-9e8a-300e7583b356 admin admin] PATH_INFO: `/v3/users/2fc6a725206d41bf9bb36da5ee7ec707` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:57.436371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ecf46-cb7e-45d8-9af2-310dea23d354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:57.436650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ecf46-cb7e-45d8-9af2-310dea23d354 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:57.436922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ecf46-cb7e-45d8-9af2-310dea23d354 admin admin] PATH_INFO: `/v3/projects/b5861ff66663413196dea3ec1af30d23` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:57.437143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3e3f2-dfd7-445d-9e8a-300e7583b356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc6a725206d41bf9bb36da5ee7ec707)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:57.438019 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ecf46-cb7e-45d8-9af2-310dea23d354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5861ff66663413196dea3ec1af30d23)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:57.440033 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3e3f2-dfd7-445d-9e8a-300e7583b356 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:57.440153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ecf46-cb7e-45d8-9af2-310dea23d354 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:57.613235 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3c3ecf46-cb7e-45d8-9af2-310dea23d354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5861ff66663413196dea3ec1af30d23', 'request_id': 'req-3c3ecf46-cb7e-45d8-9af2-310dea23d354'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.613981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7dd3e3f2-dfd7-445d-9e8a-300e7583b356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc6a725206d41bf9bb36da5ee7ec707'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.619560 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3c3ecf46-cb7e-45d8-9af2-310dea23d354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5861ff66663413196dea3ec1af30d23', 'request_id': 'req-3c3ecf46-cb7e-45d8-9af2-310dea23d354'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.696564 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7dd3e3f2-dfd7-445d-9e8a-300e7583b356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc6a725206d41bf9bb36da5ee7ec707'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.697215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7dd3e3f2-dfd7-445d-9e8a-300e7583b356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc6a725206d41bf9bb36da5ee7ec707'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.697829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7dd3e3f2-dfd7-445d-9e8a-300e7583b356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc6a725206d41bf9bb36da5ee7ec707'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.713141 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7dd3e3f2-dfd7-445d-9e8a-300e7583b356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc6a725206d41bf9bb36da5ee7ec707'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.725943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1130/2236] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:57 2022] DELETE /identity/v3/users/2fc6a725206d41bf9bb36da5ee7ec707 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:57.801888 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3c3ecf46-cb7e-45d8-9af2-310dea23d354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5861ff66663413196dea3ec1af30d23', 'request_id': 'req-3c3ecf46-cb7e-45d8-9af2-310dea23d354'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.813775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3c3ecf46-cb7e-45d8-9af2-310dea23d354 admin admin] The token cache is being invalidate because project b5861ff66663413196dea3ec1af30d23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:57.814129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3c3ecf46-cb7e-45d8-9af2-310dea23d354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.814799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3c3ecf46-cb7e-45d8-9af2-310dea23d354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:57.816326 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1107/2237] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:57 2022] DELETE /identity/v3/projects/b5861ff66663413196dea3ec1af30d23 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:57.821727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca1ea6-44d2-44a9-bbee-ba02c50bcd31 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:57.822008 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca1ea6-44d2-44a9-bbee-ba02c50bcd31 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:57.822276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca1ea6-44d2-44a9-bbee-ba02c50bcd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:57.842117 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b3ca1ea6-44d2-44a9-bbee-ba02c50bcd31 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:57.848101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b3ca1ea6-44d2-44a9-bbee-ba02c50bcd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:57.892684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:57.906688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1131/2238] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:57.914781 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dfda8-232e-4554-a7aa-736971927fe0 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:57.915126 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dfda8-232e-4554-a7aa-736971927fe0 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:57.915406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dfda8-232e-4554-a7aa-736971927fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:57.937165 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-e35dfda8-232e-4554-a7aa-736971927fe0 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:57.938210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e35dfda8-232e-4554-a7aa-736971927fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:57.949687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1108/2239] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:57.955105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca1ea6-44d2-44a9-bbee-ba02c50bcd31 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:57.965739 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca1ea6-44d2-44a9-bbee-ba02c50bcd31 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:57.970669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc91d87-7d17-4002-b645-3004c7f82e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:57.971446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc91d87-7d17-4002-b645-3004c7f82e76 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:57.971819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc91d87-7d17-4002-b645-3004c7f82e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:57.972223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc91d87-7d17-4002-b645-3004c7f82e76 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:57.973779 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc91d87-7d17-4002-b645-3004c7f82e76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:57:57.973779 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc91d87-7d17-4002-b645-3004c7f82e76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:57.975772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc91d87-7d17-4002-b645-3004c7f82e76 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:57.978298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:57.993086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1132/2240] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:57:57.994254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dfda8-232e-4554-a7aa-736971927fe0 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:58.003400 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dfda8-232e-4554-a7aa-736971927fe0 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:58.009558 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d547fcc-8650-412f-b6a9-c678746e9cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:58.014408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547fcc-8650-412f-b6a9-c678746e9cad admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:58.014408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547fcc-8650-412f-b6a9-c678746e9cad admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:58.014408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547fcc-8650-412f-b6a9-c678746e9cad admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:58.014408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d547fcc-8650-412f-b6a9-c678746e9cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:58.014408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d547fcc-8650-412f-b6a9-c678746e9cad admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:58.137685 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1109/2241] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:57:57 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:58.144470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc91d87-7d17-4002-b645-3004c7f82e76 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:58.154951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc91d87-7d17-4002-b645-3004c7f82e76 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:58.169345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d547fcc-8650-412f-b6a9-c678746e9cad admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:58.173804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2d547fcc-8650-412f-b6a9-c678746e9cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:58.226751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37d314-f4e8-435d-b5eb-8f718849758b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:58.227359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37d314-f4e8-435d-b5eb-8f718849758b admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:58.227630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37d314-f4e8-435d-b5eb-8f718849758b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:58.227896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37d314-f4e8-435d-b5eb-8f718849758b admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:58.228717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37d314-f4e8-435d-b5eb-8f718849758b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:58.230517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37d314-f4e8-435d-b5eb-8f718849758b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:58.232102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:58.252909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d547fcc-8650-412f-b6a9-c678746e9cad admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:58.258333 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:58.259084 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:58.259269 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:58.259716 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] PATH_INFO: `/v3/projects/2246860985904ff8a0a6afdd06e8bef5` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:58.260830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2246860985904ff8a0a6afdd06e8bef5)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:58.262749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:58.379889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1133/2242] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:57:58 2022] POST /identity/v3/users => generated 287 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:57:58.383981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37d314-f4e8-435d-b5eb-8f718849758b admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:58.393407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37d314-f4e8-435d-b5eb-8f718849758b admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:58.399843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4d161-df7f-4fa3-8eb7-19523f230536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:58.399843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4d161-df7f-4fa3-8eb7-19523f230536 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:58.400093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4d161-df7f-4fa3-8eb7-19523f230536 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:58.400353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4d161-df7f-4fa3-8eb7-19523f230536 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:58.401089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4d161-df7f-4fa3-8eb7-19523f230536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:58.402752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4d161-df7f-4fa3-8eb7-19523f230536 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:58.413890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1134/2243] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:58.417884 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4d161-df7f-4fa3-8eb7-19523f230536 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:58.421895 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2246860985904ff8a0a6afdd06e8bef5', 'request_id': 'req-6d3b3e7e-f117-41ab-b435-5fee1207a572'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:58.428348 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4d161-df7f-4fa3-8eb7-19523f230536 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:58.429023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2246860985904ff8a0a6afdd06e8bef5', 'request_id': 'req-6d3b3e7e-f117-41ab-b435-5fee1207a572'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:58.485668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c395f02-f6ba-4797-b1c0-ede299742f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:58.486353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c395f02-f6ba-4797-b1c0-ede299742f2e admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:58.486666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c395f02-f6ba-4797-b1c0-ede299742f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:58.486935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c395f02-f6ba-4797-b1c0-ede299742f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:58.487568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c395f02-f6ba-4797-b1c0-ede299742f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:58.489293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c395f02-f6ba-4797-b1c0-ede299742f2e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:58.492627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2246860985904ff8a0a6afdd06e8bef5', 'request_id': 'req-6d3b3e7e-f117-41ab-b435-5fee1207a572'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:58.502211 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1135/2244] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:57:58.503330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c395f02-f6ba-4797-b1c0-ede299742f2e admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:58.505769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] The token cache is being invalidate because project 2246860985904ff8a0a6afdd06e8bef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:57:58.506342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:58.507314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:58.509501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1110/2245] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:58 2022] DELETE /identity/v3/projects/2246860985904ff8a0a6afdd06e8bef5 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:58.511163 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:58.513271 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:58.515477 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:58.517350 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c395f02-f6ba-4797-b1c0-ede299742f2e admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:58.523498 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0450de-3744-41bb-b890-b1da1308d0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:58.524159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0450de-3744-41bb-b890-b1da1308d0a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:58.524440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0450de-3744-41bb-b890-b1da1308d0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:58.526287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0450de-3744-41bb-b890-b1da1308d0a7 admin admin] PATH_INFO: `/v3/projects/690ab9e0275d4b129fc0fe9e3d57d25b/users/a517c63b5dcd46dbbd034e7be4415c7c/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:58.536623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b3e7e-f117-41ab-b435-5fee1207a572 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:58.541687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b12965-d38a-4f40-ad24-f5647c1972d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:58.542595 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b12965-d38a-4f40-ad24-f5647c1972d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:58.543060 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b12965-d38a-4f40-ad24-f5647c1972d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:58.543637 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b12965-d38a-4f40-ad24-f5647c1972d3 admin admin] PATH_INFO: `/v3/users/271e3dc8558547949f90e566211019e7` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:58.545069 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b12965-d38a-4f40-ad24-f5647c1972d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=271e3dc8558547949f90e566211019e7)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:58.545445 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0450de-3744-41bb-b890-b1da1308d0a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=690ab9e0275d4b129fc0fe9e3d57d25b, user_id=a517c63b5dcd46dbbd034e7be4415c7c, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:58.548002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b12965-d38a-4f40-ad24-f5647c1972d3 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:58.549666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0450de-3744-41bb-b890-b1da1308d0a7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:58.617183 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1136/2246] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:57:58 2022] PUT /identity/v3/projects/690ab9e0275d4b129fc0fe9e3d57d25b/users/a517c63b5dcd46dbbd034e7be4415c7c/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:57:58.628736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f170-6e9e-4657-8199-eac64871786d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:58.629586 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f170-6e9e-4657-8199-eac64871786d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:58.630438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f170-6e9e-4657-8199-eac64871786d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:58.659928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d9f1f170-6e9e-4657-8199-eac64871786d None None] MFA Rules not processed for user `a517c63b5dcd46dbbd034e7be4415c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:58.661358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d9f1f170-6e9e-4657-8199-eac64871786d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:58.727973 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1137/2247] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:57:58.739729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1f170-6e9e-4657-8199-eac64871786d None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:58.741126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d9f1f170-6e9e-4657-8199-eac64871786d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:58.750837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-95b12965-d38a-4f40-ad24-f5647c1972d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '271e3dc8558547949f90e566211019e7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:58.801938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1f170-6e9e-4657-8199-eac64871786d None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:58.805422 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ca089-ab13-41d5-871b-1cebb183d65c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:58.806092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ca089-ab13-41d5-871b-1cebb183d65c service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:58.806376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ca089-ab13-41d5-871b-1cebb183d65c service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:58.806684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ca089-ab13-41d5-871b-1cebb183d65c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:58.820860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ca089-ab13-41d5-871b-1cebb183d65c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:58.822295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ca089-ab13-41d5-871b-1cebb183d65c service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:58.831703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1138/2248] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:58.835776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-95b12965-d38a-4f40-ad24-f5647c1972d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '271e3dc8558547949f90e566211019e7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:58.838667 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-95b12965-d38a-4f40-ad24-f5647c1972d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '271e3dc8558547949f90e566211019e7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:58.838667 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-95b12965-d38a-4f40-ad24-f5647c1972d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '271e3dc8558547949f90e566211019e7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:58.846886 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-95b12965-d38a-4f40-ad24-f5647c1972d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '271e3dc8558547949f90e566211019e7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:58.859950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ca089-ab13-41d5-871b-1cebb183d65c service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:58.861447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-166ca089-ab13-41d5-871b-1cebb183d65c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:58.868001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1111/2249] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:58 2022] DELETE /identity/v3/users/271e3dc8558547949f90e566211019e7 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:58.947954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ca089-ab13-41d5-871b-1cebb183d65c service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:59.024040 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:59.024695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:59.024972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:59.025351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] PATH_INFO: `/v3/users/a517c63b5dcd46dbbd034e7be4415c7c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:59.026457 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a517c63b5dcd46dbbd034e7be4415c7c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:59.028436 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:59.297241 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a517c63b5dcd46dbbd034e7be4415c7c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:59.389591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a517c63b5dcd46dbbd034e7be4415c7c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:59.390219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a517c63b5dcd46dbbd034e7be4415c7c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:59.390849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a517c63b5dcd46dbbd034e7be4415c7c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:59.405015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a517c63b5dcd46dbbd034e7be4415c7c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:57:59.417229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1139/2250] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:57:58 2022] DELETE /identity/v3/users/a517c63b5dcd46dbbd034e7be4415c7c => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:57:59.421746 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7959f5b-04f4-48c5-9fe7-a71841a069d5 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:59.422686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7959f5b-04f4-48c5-9fe7-a71841a069d5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:59.423120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7959f5b-04f4-48c5-9fe7-a71841a069d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:59.444048 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-f7959f5b-04f4-48c5-9fe7-a71841a069d5 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:57:59.445510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f7959f5b-04f4-48c5-9fe7-a71841a069d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:59.506848 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:59.510836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1112/2251] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:57:59.520694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:59.522204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:59.584207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb4996-8112-406f-bdd7-a775530971d5 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:59.587859 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797b842-69dc-4838-9dc0-1df52ce81b33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:59.588506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797b842-69dc-4838-9dc0-1df52ce81b33 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:59.588888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797b842-69dc-4838-9dc0-1df52ce81b33 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:59.589162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797b842-69dc-4838-9dc0-1df52ce81b33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:59.600864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797b842-69dc-4838-9dc0-1df52ce81b33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:59.602338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797b842-69dc-4838-9dc0-1df52ce81b33 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:57:59.614405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1140/2252] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:57:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:57:59.733184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:57:59.827441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7959f5b-04f4-48c5-9fe7-a71841a069d5 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:57:59.829827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f7959f5b-04f4-48c5-9fe7-a71841a069d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:57:59.849334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7959f5b-04f4-48c5-9fe7-a71841a069d5 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:57:59.854974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:57:59.855619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:57:59.855902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:57:59.856182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] PATH_INFO: `/v3/projects/68931eadcdac47c98c71285db2775be5` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:57:59.857328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68931eadcdac47c98c71285db2775be5)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:57:59.869489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:00.009495 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68931eadcdac47c98c71285db2775be5', 'request_id': 'req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:00.021675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68931eadcdac47c98c71285db2775be5', 'request_id': 'req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:00.125248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68931eadcdac47c98c71285db2775be5', 'request_id': 'req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:00.137254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] The token cache is being invalidate because project 68931eadcdac47c98c71285db2775be5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:58:00.137544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:00.138193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:00.139702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1113/2253] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:57:59 2022] DELETE /identity/v3/projects/68931eadcdac47c98c71285db2775be5 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:00.146188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b90784-8913-454e-80de-e24e264553db None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.146705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b90784-8913-454e-80de-e24e264553db None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.147053 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b90784-8913-454e-80de-e24e264553db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.169121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:00.172105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:00.172753 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-a3b90784-8913-454e-80de-e24e264553db None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:00.174808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a3b90784-8913-454e-80de-e24e264553db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:00.257457 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d351f60-d4ce-4fd7-9943-92d1814e6ba9 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:00.264254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:00.265502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.270625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.270625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5 admin admin] PATH_INFO: `/v3/projects/690ab9e0275d4b129fc0fe9e3d57d25b` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.270625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=690ab9e0275d4b129fc0fe9e3d57d25b)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:00.270625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:00.283256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1141/2254] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:00.291960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218a102-517c-420e-b49b-e24db6f592ae None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.292460 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218a102-517c-420e-b49b-e24db6f592ae None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.292742 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218a102-517c-420e-b49b-e24db6f592ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.326539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-1218a102-517c-420e-b49b-e24db6f592ae None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:00.328095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1218a102-517c-420e-b49b-e24db6f592ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:00.338010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1142/2255] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:58:00.342593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218a102-517c-420e-b49b-e24db6f592ae None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:00.353080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218a102-517c-420e-b49b-e24db6f592ae None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:00.358824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d9290-da90-44d7-bf32-3ca84455e80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:00.359436 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d9290-da90-44d7-bf32-3ca84455e80a admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.359717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d9290-da90-44d7-bf32-3ca84455e80a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.359993 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d9290-da90-44d7-bf32-3ca84455e80a admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.360651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d9290-da90-44d7-bf32-3ca84455e80a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:58:00.360961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d9290-da90-44d7-bf32-3ca84455e80a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:00.362591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d9290-da90-44d7-bf32-3ca84455e80a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:00.374992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1143/2256] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:58:00.379372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d9290-da90-44d7-bf32-3ca84455e80a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:00.389265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d9290-da90-44d7-bf32-3ca84455e80a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:00.397677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58034eb6-a0d7-469a-b145-0eac95347c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:00.398300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58034eb6-a0d7-469a-b145-0eac95347c26 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.398617 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58034eb6-a0d7-469a-b145-0eac95347c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.398891 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58034eb6-a0d7-469a-b145-0eac95347c26 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.399666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58034eb6-a0d7-469a-b145-0eac95347c26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:00.418603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58034eb6-a0d7-469a-b145-0eac95347c26 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:00.423176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690ab9e0275d4b129fc0fe9e3d57d25b', 'request_id': 'req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:00.453479 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690ab9e0275d4b129fc0fe9e3d57d25b', 'request_id': 'req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:00.516439 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690ab9e0275d4b129fc0fe9e3d57d25b', 'request_id': 'req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:00.529343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1144/2257] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:58:00 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:58:00.530847 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58034eb6-a0d7-469a-b145-0eac95347c26 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:00.531029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5 admin admin] The token cache is being invalidate because project 690ab9e0275d4b129fc0fe9e3d57d25b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:58:00.531415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:00.532522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d7d5a9b5-3b7b-4d88-8e00-bdd2f7c012f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:00.534510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1114/2258] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:58:00 2022] DELETE /identity/v3/projects/690ab9e0275d4b129fc0fe9e3d57d25b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:00.544576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cf91c-2ed3-414f-85f0-7e427f1ea3d7 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.545165 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cf91c-2ed3-414f-85f0-7e427f1ea3d7 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.545613 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cf91c-2ed3-414f-85f0-7e427f1ea3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.571903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-8b2cf91c-2ed3-414f-85f0-7e427f1ea3d7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:00.585333 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-8b2cf91c-2ed3-414f-85f0-7e427f1ea3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:00.605458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:00.630327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58034eb6-a0d7-469a-b145-0eac95347c26 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:00.636153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0723-51af-4799-ac65-7405434a500a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:00.636963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0723-51af-4799-ac65-7405434a500a admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.637239 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0723-51af-4799-ac65-7405434a500a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.637586 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0723-51af-4799-ac65-7405434a500a admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.638284 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e0723-51af-4799-ac65-7405434a500a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:00.640394 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e0723-51af-4799-ac65-7405434a500a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:00.702596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1115/2259] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:00.711845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555b76a-822e-49b0-906f-0c762dca49a4 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.712137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555b76a-822e-49b0-906f-0c762dca49a4 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.712415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555b76a-822e-49b0-906f-0c762dca49a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.732186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-2555b76a-822e-49b0-906f-0c762dca49a4 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:00.733393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2555b76a-822e-49b0-906f-0c762dca49a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:00.742046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1116/2260] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:58:00.746221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555b76a-822e-49b0-906f-0c762dca49a4 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:00.750370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1145/2261] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:58:00 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:58:00.754953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0723-51af-4799-ac65-7405434a500a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:00.756375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555b76a-822e-49b0-906f-0c762dca49a4 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:00.756582 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ee7e0723-51af-4799-ac65-7405434a500a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:00.761001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26456c-ab88-4f66-a2b7-0497bad1a4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:00.761599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26456c-ab88-4f66-a2b7-0497bad1a4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.761881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26456c-ab88-4f66-a2b7-0497bad1a4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.762155 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26456c-ab88-4f66-a2b7-0497bad1a4f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.762846 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26456c-ab88-4f66-a2b7-0497bad1a4f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:58:00.763136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26456c-ab88-4f66-a2b7-0497bad1a4f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:00.764761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26456c-ab88-4f66-a2b7-0497bad1a4f7 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:00.768516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0723-51af-4799-ac65-7405434a500a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:00.782875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1117/2262] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:58:00.783860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26456c-ab88-4f66-a2b7-0497bad1a4f7 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:00.783972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc796915-2698-451f-b405-05512613f9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:00.784571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc796915-2698-451f-b405-05512613f9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.784999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc796915-2698-451f-b405-05512613f9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.785328 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc796915-2698-451f-b405-05512613f9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.785982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc796915-2698-451f-b405-05512613f9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:00.787738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc796915-2698-451f-b405-05512613f9b4 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:00.794155 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26456c-ab88-4f66-a2b7-0497bad1a4f7 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:00.798819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1146/2263] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:58:00.802376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ceb12a-7d0c-4b1d-942e-54dedabaad93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:00.802376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceb12a-7d0c-4b1d-942e-54dedabaad93 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.802376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceb12a-7d0c-4b1d-942e-54dedabaad93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.802376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ceb12a-7d0c-4b1d-942e-54dedabaad93 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.802748 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ceb12a-7d0c-4b1d-942e-54dedabaad93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:00.803831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ceb12a-7d0c-4b1d-942e-54dedabaad93 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:00.805400 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc796915-2698-451f-b405-05512613f9b4 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:00.815419 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc796915-2698-451f-b405-05512613f9b4 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:00.821316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71287a90-bd52-48f3-9848-88f0fbb25438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:00.821476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71287a90-bd52-48f3-9848-88f0fbb25438 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.821799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71287a90-bd52-48f3-9848-88f0fbb25438 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.822077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71287a90-bd52-48f3-9848-88f0fbb25438 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.822704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71287a90-bd52-48f3-9848-88f0fbb25438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:00.824302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71287a90-bd52-48f3-9848-88f0fbb25438 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:00.836665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1147/2264] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:58:00.840596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71287a90-bd52-48f3-9848-88f0fbb25438 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:00.850772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71287a90-bd52-48f3-9848-88f0fbb25438 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:00.855368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf722c9-22e8-4d45-aefc-3f8d2b8f81f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:00.856093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf722c9-22e8-4d45-aefc-3f8d2b8f81f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.856379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf722c9-22e8-4d45-aefc-3f8d2b8f81f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.856665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf722c9-22e8-4d45-aefc-3f8d2b8f81f7 admin admin] PATH_INFO: `/v3/projects/deec9ed17a894a749c71b022c004b46e/users/e8941816770d46b98d53dedbf6f840e4/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.871732 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf722c9-22e8-4d45-aefc-3f8d2b8f81f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deec9ed17a894a749c71b022c004b46e, user_id=e8941816770d46b98d53dedbf6f840e4, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:00.874351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf722c9-22e8-4d45-aefc-3f8d2b8f81f7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:00.917999 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1118/2265] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:58:00 2022] POST /identity/v3/projects => generated 391 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:58:00.918987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ceb12a-7d0c-4b1d-942e-54dedabaad93 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:00.956278 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1148/2266] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:58:00 2022] PUT /identity/v3/projects/deec9ed17a894a749c71b022c004b46e/users/e8941816770d46b98d53dedbf6f840e4/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:58:00.967570 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa267a-c15b-4a60-8563-5cd13214b344 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:00.967999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa267a-c15b-4a60-8563-5cd13214b344 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:00.968304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa267a-c15b-4a60-8563-5cd13214b344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:00.987447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ceb12a-7d0c-4b1d-942e-54dedabaad93 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:00.998001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-dffa267a-c15b-4a60-8563-5cd13214b344 None None] MFA Rules not processed for user `e8941816770d46b98d53dedbf6f840e4`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:00.999316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-dffa267a-c15b-4a60-8563-5cd13214b344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:01.001573 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:01.050355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0940e-8411-4e0b-8cd5-8653417104f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:01.051637 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0940e-8411-4e0b-8cd5-8653417104f5 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:01.051932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0940e-8411-4e0b-8cd5-8653417104f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:01.052209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0940e-8411-4e0b-8cd5-8653417104f5 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:01.054114 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0940e-8411-4e0b-8cd5-8653417104f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:01.056129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0940e-8411-4e0b-8cd5-8653417104f5 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:01.071390 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1149/2267] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:58:01.087233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa267a-c15b-4a60-8563-5cd13214b344 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:01.088587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-dffa267a-c15b-4a60-8563-5cd13214b344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:01.123029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1119/2268] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:58:00 2022] POST /identity/v3/users => generated 296 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:58:01.126682 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0940e-8411-4e0b-8cd5-8653417104f5 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:01.137752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0940e-8411-4e0b-8cd5-8653417104f5 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:01.142629 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320659e-c822-4422-a963-e390941daf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:01.143219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320659e-c822-4422-a963-e390941daf12 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:01.143501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320659e-c822-4422-a963-e390941daf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:01.143775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320659e-c822-4422-a963-e390941daf12 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:01.144443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320659e-c822-4422-a963-e390941daf12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:01.146615 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320659e-c822-4422-a963-e390941daf12 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:01.151536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa267a-c15b-4a60-8563-5cd13214b344 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:01.156025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8165e-3479-4272-8736-9a4f16ca5af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:01.156360 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8165e-3479-4272-8736-9a4f16ca5af4 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:01.156711 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8165e-3479-4272-8736-9a4f16ca5af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:01.156995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8165e-3479-4272-8736-9a4f16ca5af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:01.160747 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1120/2269] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:58:01.161953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320659e-c822-4422-a963-e390941daf12 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:01.166163 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8165e-3479-4272-8736-9a4f16ca5af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:01.167620 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8165e-3479-4272-8736-9a4f16ca5af4 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:01.171933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320659e-c822-4422-a963-e390941daf12 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:01.177082 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1150/2270] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:58:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:58:01.177337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dab03f-a4ed-4647-a023-cb1cbc673a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:01.177934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dab03f-a4ed-4647-a023-cb1cbc673a1f admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:01.178271 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dab03f-a4ed-4647-a023-cb1cbc673a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:01.179765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dab03f-a4ed-4647-a023-cb1cbc673a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:01.179765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dab03f-a4ed-4647-a023-cb1cbc673a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:01.180872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dab03f-a4ed-4647-a023-cb1cbc673a1f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:01.192184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1121/2271] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:58:01.196472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8165e-3479-4272-8736-9a4f16ca5af4 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:01.207935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8165e-3479-4272-8736-9a4f16ca5af4 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:01.212496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5a899-3541-4bb4-80e5-1e25789e6dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:01.213139 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5a899-3541-4bb4-80e5-1e25789e6dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:01.213411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5a899-3541-4bb4-80e5-1e25789e6dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:01.213677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5a899-3541-4bb4-80e5-1e25789e6dbc admin admin] PATH_INFO: `/v3/projects/5219d434ae36439091135c08f5fa44de/users/73e2a7d3b2304d34a2f95719edfc3a81/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:01.230119 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5a899-3541-4bb4-80e5-1e25789e6dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5219d434ae36439091135c08f5fa44de, user_id=73e2a7d3b2304d34a2f95719edfc3a81, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:01.232882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5a899-3541-4bb4-80e5-1e25789e6dbc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:01.316252 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1151/2272] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:58:01 2022] PUT /identity/v3/projects/5219d434ae36439091135c08f5fa44de/users/73e2a7d3b2304d34a2f95719edfc3a81/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:01.327313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f5882-e73a-4598-a922-23450fdf26c1 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:01.327602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f5882-e73a-4598-a922-23450fdf26c1 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:01.328142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f5882-e73a-4598-a922-23450fdf26c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:01.358315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-bc5f5882-e73a-4598-a922-23450fdf26c1 None None] MFA Rules not processed for user `73e2a7d3b2304d34a2f95719edfc3a81`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:01.359651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-bc5f5882-e73a-4598-a922-23450fdf26c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:01.424318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1122/2273] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:01.435256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5a899-3541-4bb4-80e5-1e25789e6dbc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:01.436936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-aae5a899-3541-4bb4-80e5-1e25789e6dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:01.493001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5a899-3541-4bb4-80e5-1e25789e6dbc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:01.496981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d155e-fbd7-4f83-b3ad-d1658adb8679 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:01.497891 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d155e-fbd7-4f83-b3ad-d1658adb8679 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:01.498176 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d155e-fbd7-4f83-b3ad-d1658adb8679 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:01.498447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d155e-fbd7-4f83-b3ad-d1658adb8679 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:01.507901 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d155e-fbd7-4f83-b3ad-d1658adb8679 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:01.509415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d155e-fbd7-4f83-b3ad-d1658adb8679 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:01.518331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1152/2274] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:58:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:58:01.892961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:02.138723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:03.077767 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:04.372395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f5882-e73a-4598-a922-23450fdf26c1 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:04.399736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:04.434413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f5882-e73a-4598-a922-23450fdf26c1 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:04.439029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:04.439657 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:04.439940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:04.440215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] PATH_INFO: `/v3/users/e8941816770d46b98d53dedbf6f840e4` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:04.441325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8941816770d46b98d53dedbf6f840e4)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:04.444139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:04.659826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8941816770d46b98d53dedbf6f840e4'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:04.730436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8941816770d46b98d53dedbf6f840e4'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:04.731089 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8941816770d46b98d53dedbf6f840e4'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:04.731704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8941816770d46b98d53dedbf6f840e4'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:04.740130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8941816770d46b98d53dedbf6f840e4'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:04.758934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1123/2275] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:58:04 2022] DELETE /identity/v3/users/e8941816770d46b98d53dedbf6f840e4 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:04.760471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:04.763275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3985e-0e97-4612-ac15-1ff29a26194a None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:04.768387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3985e-0e97-4612-ac15-1ff29a26194a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:04.768387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3985e-0e97-4612-ac15-1ff29a26194a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:04.782677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-09b3985e-0e97-4612-ac15-1ff29a26194a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:04.783976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-09b3985e-0e97-4612-ac15-1ff29a26194a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:04.846132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1153/2276] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:58:04.856086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:04.857576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:04.918672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1c432-9760-4030-bf9b-492a29185eb8 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:04.922711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98264929-4c6c-482d-9555-33cb8ff15fd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:04.923008 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98264929-4c6c-482d-9555-33cb8ff15fd9 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:04.923461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98264929-4c6c-482d-9555-33cb8ff15fd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:04.923797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98264929-4c6c-482d-9555-33cb8ff15fd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:04.937518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98264929-4c6c-482d-9555-33cb8ff15fd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:04.939047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98264929-4c6c-482d-9555-33cb8ff15fd9 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:04.949403 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1124/2277] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:58:04.972956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3985e-0e97-4612-ac15-1ff29a26194a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:04.974993 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-09b3985e-0e97-4612-ac15-1ff29a26194a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:04.986896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3985e-0e97-4612-ac15-1ff29a26194a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:04.993430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:04.994047 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:04.994320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:04.994667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] PATH_INFO: `/v3/users/feea737a0e5f44c3a97bd9a859f71e67` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:04.995848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feea737a0e5f44c3a97bd9a859f71e67)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:04.997795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:05.221033 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feea737a0e5f44c3a97bd9a859f71e67'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:05.287411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feea737a0e5f44c3a97bd9a859f71e67'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:05.288226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feea737a0e5f44c3a97bd9a859f71e67'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:05.288964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feea737a0e5f44c3a97bd9a859f71e67'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:05.302572 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feea737a0e5f44c3a97bd9a859f71e67'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:05.315272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1154/2278] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:58:04 2022] DELETE /identity/v3/users/feea737a0e5f44c3a97bd9a859f71e67 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:05.320499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a37bda-f9c7-4c39-a09d-d2aab2a8ac3c None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:05.327617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a37bda-f9c7-4c39-a09d-d2aab2a8ac3c None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:05.327617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a37bda-f9c7-4c39-a09d-d2aab2a8ac3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:05.345163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-20a37bda-f9c7-4c39-a09d-d2aab2a8ac3c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:05.346587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-20a37bda-f9c7-4c39-a09d-d2aab2a8ac3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:05.409505 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1125/2279] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:05.423356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:05.423356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:05.425197 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a37bda-f9c7-4c39-a09d-d2aab2a8ac3c None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:05.426693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-20a37bda-f9c7-4c39-a09d-d2aab2a8ac3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:05.439455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:05.445799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a37bda-f9c7-4c39-a09d-d2aab2a8ac3c None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:05.455124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ae331-7813-46c8-b42d-aea851a2e666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:05.455124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ae331-7813-46c8-b42d-aea851a2e666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:05.455124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ae331-7813-46c8-b42d-aea851a2e666 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:05.455124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ae331-7813-46c8-b42d-aea851a2e666 admin admin] PATH_INFO: `/v3/projects/deec9ed17a894a749c71b022c004b46e` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:05.455662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ae331-7813-46c8-b42d-aea851a2e666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deec9ed17a894a749c71b022c004b46e)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:05.455662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ae331-7813-46c8-b42d-aea851a2e666 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:05.495792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c07b1-36cb-40d7-a8d4-f1e71949e1d5 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:05.499613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e807cc-95e6-40de-8dac-90dc6bdbcf14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:05.500211 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e807cc-95e6-40de-8dac-90dc6bdbcf14 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:05.500483 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e807cc-95e6-40de-8dac-90dc6bdbcf14 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:05.500750 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e807cc-95e6-40de-8dac-90dc6bdbcf14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:05.510294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e807cc-95e6-40de-8dac-90dc6bdbcf14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:05.511737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e807cc-95e6-40de-8dac-90dc6bdbcf14 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:05.520992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1155/2280] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:58:05.568953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-159ae331-7813-46c8-b42d-aea851a2e666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deec9ed17a894a749c71b022c004b46e', 'request_id': 'req-159ae331-7813-46c8-b42d-aea851a2e666'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:05.575641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-159ae331-7813-46c8-b42d-aea851a2e666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deec9ed17a894a749c71b022c004b46e', 'request_id': 'req-159ae331-7813-46c8-b42d-aea851a2e666'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:05.673323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-159ae331-7813-46c8-b42d-aea851a2e666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deec9ed17a894a749c71b022c004b46e', 'request_id': 'req-159ae331-7813-46c8-b42d-aea851a2e666'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:05.682679 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:05.687213 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-159ae331-7813-46c8-b42d-aea851a2e666 admin admin] The token cache is being invalidate because project deec9ed17a894a749c71b022c004b46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:58:05.689494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-159ae331-7813-46c8-b42d-aea851a2e666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:05.689494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-159ae331-7813-46c8-b42d-aea851a2e666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:05.693244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1126/2281] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:58:05 2022] DELETE /identity/v3/projects/deec9ed17a894a749c71b022c004b46e => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:05.694986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cf254-b2c2-4efa-9893-9002c3477c82 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:05.695332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cf254-b2c2-4efa-9893-9002c3477c82 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:05.695667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cf254-b2c2-4efa-9893-9002c3477c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:05.715779 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-347cf254-b2c2-4efa-9893-9002c3477c82 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:05.719908 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-347cf254-b2c2-4efa-9893-9002c3477c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:05.783135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1156/2282] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:05.791514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752f3c2-ea57-4681-9eb1-feef69fb6a67 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:05.791868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752f3c2-ea57-4681-9eb1-feef69fb6a67 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:05.792266 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752f3c2-ea57-4681-9eb1-feef69fb6a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:05.813483 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-a752f3c2-ea57-4681-9eb1-feef69fb6a67 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:05.814810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a752f3c2-ea57-4681-9eb1-feef69fb6a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:05.825128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1127/2283] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:05.832944 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cf254-b2c2-4efa-9893-9002c3477c82 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:05.846636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cf254-b2c2-4efa-9893-9002c3477c82 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:05.853843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09155f6b-1b35-4899-972e-4442d62a2e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:05.859688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09155f6b-1b35-4899-972e-4442d62a2e76 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:05.859688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09155f6b-1b35-4899-972e-4442d62a2e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:05.859688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09155f6b-1b35-4899-972e-4442d62a2e76 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:05.859688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09155f6b-1b35-4899-972e-4442d62a2e76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:58:05.859688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09155f6b-1b35-4899-972e-4442d62a2e76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:05.859688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09155f6b-1b35-4899-972e-4442d62a2e76 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:05.870443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1157/2284] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:58:05.874856 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752f3c2-ea57-4681-9eb1-feef69fb6a67 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:05.884113 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752f3c2-ea57-4681-9eb1-feef69fb6a67 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:05.888694 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5e0a9-6e86-42fc-9551-fd382eed4132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:05.890338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5e0a9-6e86-42fc-9551-fd382eed4132 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:05.890338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5e0a9-6e86-42fc-9551-fd382eed4132 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:05.890338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5e0a9-6e86-42fc-9551-fd382eed4132 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:05.890633 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5e0a9-6e86-42fc-9551-fd382eed4132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:05.892407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5e0a9-6e86-42fc-9551-fd382eed4132 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:05.943283 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09155f6b-1b35-4899-972e-4442d62a2e76 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:05.945486 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-09155f6b-1b35-4899-972e-4442d62a2e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:05.958725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09155f6b-1b35-4899-972e-4442d62a2e76 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:05.963735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa119cb3-011b-4a53-8890-fa6886e768f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:05.964365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa119cb3-011b-4a53-8890-fa6886e768f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:05.964645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa119cb3-011b-4a53-8890-fa6886e768f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:05.964922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa119cb3-011b-4a53-8890-fa6886e768f7 admin admin] PATH_INFO: `/v3/projects/c8c7a97f6135463b924e15fec21f6b43` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:05.966144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa119cb3-011b-4a53-8890-fa6886e768f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c7a97f6135463b924e15fec21f6b43)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:05.968078 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa119cb3-011b-4a53-8890-fa6886e768f7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:05.982576 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:06.029412 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1128/2285] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:58:05 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:58:06.033278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5e0a9-6e86-42fc-9551-fd382eed4132 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:06.081999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-aa119cb3-011b-4a53-8890-fa6886e768f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c7a97f6135463b924e15fec21f6b43', 'request_id': 'req-aa119cb3-011b-4a53-8890-fa6886e768f7'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:06.090238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-aa119cb3-011b-4a53-8890-fa6886e768f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c7a97f6135463b924e15fec21f6b43', 'request_id': 'req-aa119cb3-011b-4a53-8890-fa6886e768f7'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:06.096847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5e0a9-6e86-42fc-9551-fd382eed4132 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:06.153121 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-aa119cb3-011b-4a53-8890-fa6886e768f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c7a97f6135463b924e15fec21f6b43', 'request_id': 'req-aa119cb3-011b-4a53-8890-fa6886e768f7'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:06.154009 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a9043-6ea4-4571-a812-bab5b8771472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:06.154668 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a9043-6ea4-4571-a812-bab5b8771472 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.154964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a9043-6ea4-4571-a812-bab5b8771472 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.155237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a9043-6ea4-4571-a812-bab5b8771472 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.155853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a9043-6ea4-4571-a812-bab5b8771472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:06.157729 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a9043-6ea4-4571-a812-bab5b8771472 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:06.165286 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-aa119cb3-011b-4a53-8890-fa6886e768f7 admin admin] The token cache is being invalidate because project c8c7a97f6135463b924e15fec21f6b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:58:06.165573 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-aa119cb3-011b-4a53-8890-fa6886e768f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:06.166380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-aa119cb3-011b-4a53-8890-fa6886e768f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:06.169009 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1158/2286] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:58:05 2022] DELETE /identity/v3/projects/c8c7a97f6135463b924e15fec21f6b43 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:06.173363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce9064-f970-438c-a557-99498ff6dfe2 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.173643 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce9064-f970-438c-a557-99498ff6dfe2 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.173913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce9064-f970-438c-a557-99498ff6dfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.193138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-f6ce9064-f970-438c-a557-99498ff6dfe2 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:06.194297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f6ce9064-f970-438c-a557-99498ff6dfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:06.202475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1159/2287] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:58:06.210613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bd0bf-b990-42fd-b78d-d1b6434aa5de None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.210896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bd0bf-b990-42fd-b78d-d1b6434aa5de None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.211171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bd0bf-b990-42fd-b78d-d1b6434aa5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.230937 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-514bd0bf-b990-42fd-b78d-d1b6434aa5de None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:06.232169 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-514bd0bf-b990-42fd-b78d-d1b6434aa5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:06.235998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1129/2288] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:58:06 2022] POST /identity/v3/users => generated 285 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:58:06.238872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a9043-6ea4-4571-a812-bab5b8771472 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:06.239626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-8d6a9043-6ea4-4571-a812-bab5b8771472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:06.244330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1160/2289] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:58:06.248405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bd0bf-b990-42fd-b78d-d1b6434aa5de None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:06.252426 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a9043-6ea4-4571-a812-bab5b8771472 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:06.257343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4b07d-b3fa-40bc-ab4a-b908381f8e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:06.257933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bd0bf-b990-42fd-b78d-d1b6434aa5de None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:06.258044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4b07d-b3fa-40bc-ab4a-b908381f8e63 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.258278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4b07d-b3fa-40bc-ab4a-b908381f8e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.258582 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4b07d-b3fa-40bc-ab4a-b908381f8e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.259243 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4b07d-b3fa-40bc-ab4a-b908381f8e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:06.260950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4b07d-b3fa-40bc-ab4a-b908381f8e63 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:06.262813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aea35-d9d3-4b05-bc24-2361e2c34fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:06.263402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aea35-d9d3-4b05-bc24-2361e2c34fcc admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.263738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aea35-d9d3-4b05-bc24-2361e2c34fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.264011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aea35-d9d3-4b05-bc24-2361e2c34fcc admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.264689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9aea35-d9d3-4b05-bc24-2361e2c34fcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:58:06.264995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9aea35-d9d3-4b05-bc24-2361e2c34fcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:06.266677 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9aea35-d9d3-4b05-bc24-2361e2c34fcc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:06.271716 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1130/2290] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:58:06.276085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4b07d-b3fa-40bc-ab4a-b908381f8e63 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:06.281084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1161/2291] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:58:06.283409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aea35-d9d3-4b05-bc24-2361e2c34fcc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:06.285926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4b07d-b3fa-40bc-ab4a-b908381f8e63 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:06.291935 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afe3d3-c313-49d8-ae99-de32ca051714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:06.292677 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afe3d3-c313-49d8-ae99-de32ca051714 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.292960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afe3d3-c313-49d8-ae99-de32ca051714 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.293235 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afe3d3-c313-49d8-ae99-de32ca051714 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.296579 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afe3d3-c313-49d8-ae99-de32ca051714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:06.296683 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aea35-d9d3-4b05-bc24-2361e2c34fcc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:06.296774 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afe3d3-c313-49d8-ae99-de32ca051714 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:06.299550 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61851204-7851-4ec0-b7c6-b407a6304bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:06.300258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61851204-7851-4ec0-b7c6-b407a6304bdc admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.300564 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61851204-7851-4ec0-b7c6-b407a6304bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.300838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61851204-7851-4ec0-b7c6-b407a6304bdc admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.301460 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61851204-7851-4ec0-b7c6-b407a6304bdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:06.303553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61851204-7851-4ec0-b7c6-b407a6304bdc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:06.306934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1131/2292] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:58:06.310995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afe3d3-c313-49d8-ae99-de32ca051714 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:06.320798 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afe3d3-c313-49d8-ae99-de32ca051714 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:06.325169 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:06.328156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d34292-bbb5-416d-8cd9-801b7e4b9f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:06.328821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d34292-bbb5-416d-8cd9-801b7e4b9f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.329096 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d34292-bbb5-416d-8cd9-801b7e4b9f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.329389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d34292-bbb5-416d-8cd9-801b7e4b9f2b admin admin] PATH_INFO: `/v3/projects/f0dcc96f9565435eabccde5ac25dbcd3/users/33a9a55dd16b49dbb97e6b25bcc1e438/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.344322 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d34292-bbb5-416d-8cd9-801b7e4b9f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0dcc96f9565435eabccde5ac25dbcd3, user_id=33a9a55dd16b49dbb97e6b25bcc1e438, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:06.346979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d34292-bbb5-416d-8cd9-801b7e4b9f2b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:06.403098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1162/2293] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:58:06 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:58:06.407048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61851204-7851-4ec0-b7c6-b407a6304bdc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:06.447769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1132/2294] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:58:06 2022] PUT /identity/v3/projects/f0dcc96f9565435eabccde5ac25dbcd3/users/33a9a55dd16b49dbb97e6b25bcc1e438/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:58:06.458298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05c3c5-5f94-4b41-8a7f-f12ba491c537 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.458874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05c3c5-5f94-4b41-8a7f-f12ba491c537 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.459251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05c3c5-5f94-4b41-8a7f-f12ba491c537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.475770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61851204-7851-4ec0-b7c6-b407a6304bdc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:06.491087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-6f05c3c5-5f94-4b41-8a7f-f12ba491c537 None None] MFA Rules not processed for user `33a9a55dd16b49dbb97e6b25bcc1e438`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:06.492570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6f05c3c5-5f94-4b41-8a7f-f12ba491c537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:06.542414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c2fa2-947d-49b6-82ec-c4f07b1a5350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:06.542991 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c2fa2-947d-49b6-82ec-c4f07b1a5350 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.543384 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c2fa2-947d-49b6-82ec-c4f07b1a5350 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.543596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c2fa2-947d-49b6-82ec-c4f07b1a5350 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.544144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c2fa2-947d-49b6-82ec-c4f07b1a5350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:06.546039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c2fa2-947d-49b6-82ec-c4f07b1a5350 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:06.571707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1133/2295] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:58:06.583562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05c3c5-5f94-4b41-8a7f-f12ba491c537 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:06.590876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6f05c3c5-5f94-4b41-8a7f-f12ba491c537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:06.642125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05c3c5-5f94-4b41-8a7f-f12ba491c537 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:06.647874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92999860-24ea-4ce4-9677-414b7f8c8151 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff8fcb7fa2740bbb801badf800d2d6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:06.648486 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92999860-24ea-4ce4-9677-414b7f8c8151 service glance] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.648767 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92999860-24ea-4ce4-9677-414b7f8c8151 service glance] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.649045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92999860-24ea-4ce4-9677-414b7f8c8151 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.662534 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1163/2296] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:58:06 2022] POST /identity/v3/users => generated 287 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:58:06.662726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92999860-24ea-4ce4-9677-414b7f8c8151 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:06.662726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92999860-24ea-4ce4-9677-414b7f8c8151 service glance] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:06.663616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c2fa2-947d-49b6-82ec-c4f07b1a5350 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:06.671645 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1134/2297] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:58:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:58:06.674069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c2fa2-947d-49b6-82ec-c4f07b1a5350 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:06.677620 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:06.684650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f043ac-a80e-4fa1-8fc2-b5d2f361ebef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:06.684650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f043ac-a80e-4fa1-8fc2-b5d2f361ebef admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.684650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f043ac-a80e-4fa1-8fc2-b5d2f361ebef admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.684650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f043ac-a80e-4fa1-8fc2-b5d2f361ebef admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.685162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f043ac-a80e-4fa1-8fc2-b5d2f361ebef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:06.686936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f043ac-a80e-4fa1-8fc2-b5d2f361ebef admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:06.698966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1164/2298] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:58:06.702820 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92999860-24ea-4ce4-9677-414b7f8c8151 service glance] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:06.714589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92999860-24ea-4ce4-9677-414b7f8c8151 service glance] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:06.719195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68624ca-5872-4c37-bcc5-820dabd98847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:06.719806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68624ca-5872-4c37-bcc5-820dabd98847 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.720093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68624ca-5872-4c37-bcc5-820dabd98847 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.720356 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68624ca-5872-4c37-bcc5-820dabd98847 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.720989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68624ca-5872-4c37-bcc5-820dabd98847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:06.722634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68624ca-5872-4c37-bcc5-820dabd98847 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:06.733371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1135/2299] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:58:06.737729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f043ac-a80e-4fa1-8fc2-b5d2f361ebef admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:06.749209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f043ac-a80e-4fa1-8fc2-b5d2f361ebef admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:06.754540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb24df-15cb-486c-b820-969eb991a778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:06.755740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb24df-15cb-486c-b820-969eb991a778 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.756020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb24df-15cb-486c-b820-969eb991a778 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.756293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb24df-15cb-486c-b820-969eb991a778 admin admin] PATH_INFO: `/v3/projects/f19f488a52f64b53b4b194364e1db3a9/users/9742feebb05246d59d80bbb9e4f6a2ca/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.772391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb24df-15cb-486c-b820-969eb991a778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19f488a52f64b53b4b194364e1db3a9, user_id=9742feebb05246d59d80bbb9e4f6a2ca, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:06.775297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb24df-15cb-486c-b820-969eb991a778 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:06.857354 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1165/2300] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:58:06 2022] PUT /identity/v3/projects/f19f488a52f64b53b4b194364e1db3a9/users/9742feebb05246d59d80bbb9e4f6a2ca/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:06.870871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26803c2f-463f-4d00-a3ac-8ed86a735cca None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:06.871523 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26803c2f-463f-4d00-a3ac-8ed86a735cca None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:06.871899 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26803c2f-463f-4d00-a3ac-8ed86a735cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:06.907951 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-26803c2f-463f-4d00-a3ac-8ed86a735cca None None] MFA Rules not processed for user `9742feebb05246d59d80bbb9e4f6a2ca`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:06.909252 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-26803c2f-463f-4d00-a3ac-8ed86a735cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:06.972342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:06.993793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1136/2301] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:07.004656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb24df-15cb-486c-b820-969eb991a778 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:07.008052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-bbbb24df-15cb-486c-b820-969eb991a778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:07.065234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb24df-15cb-486c-b820-969eb991a778 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:07.069813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1e693-dbf8-4e4a-9479-4adbc5b0f3ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:07.070449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1e693-dbf8-4e4a-9479-4adbc5b0f3ad service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:07.070789 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1e693-dbf8-4e4a-9479-4adbc5b0f3ad service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:07.071084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1e693-dbf8-4e4a-9479-4adbc5b0f3ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:07.079613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1e693-dbf8-4e4a-9479-4adbc5b0f3ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:07.081001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1e693-dbf8-4e4a-9479-4adbc5b0f3ad service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:07.090410 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1166/2302] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:58:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:58:07.504213 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:07.955358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:08.394083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:08.601131 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:08.669715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:08.692845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:08.784023 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:08.898440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:09.018512 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:09.104174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:09.189579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:09.275517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:09.407051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:09.492620 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:19.216010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=70 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:19.549321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=14 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:28.439538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26803c2f-463f-4d00-a3ac-8ed86a735cca None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:28.441080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-26803c2f-463f-4d00-a3ac-8ed86a735cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:28.519190 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26803c2f-463f-4d00-a3ac-8ed86a735cca None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:28.525253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:28.525902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:28.526182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:28.526457 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] PATH_INFO: `/v3/users/73e2a7d3b2304d34a2f95719edfc3a81` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:28.527709 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e2a7d3b2304d34a2f95719edfc3a81)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:28.529902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:28.758917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e2a7d3b2304d34a2f95719edfc3a81'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:28.872879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e2a7d3b2304d34a2f95719edfc3a81'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:28.873496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e2a7d3b2304d34a2f95719edfc3a81'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:28.874100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e2a7d3b2304d34a2f95719edfc3a81'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:28.882781 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e2a7d3b2304d34a2f95719edfc3a81'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:28.901672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1137/2303] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:58:28 2022] DELETE /identity/v3/users/73e2a7d3b2304d34a2f95719edfc3a81 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:28.906030 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a75e7b-1266-4b59-920b-bc1c2fbc751e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:28.906399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a75e7b-1266-4b59-920b-bc1c2fbc751e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:28.906679 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a75e7b-1266-4b59-920b-bc1c2fbc751e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:28.931704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-40a75e7b-1266-4b59-920b-bc1c2fbc751e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:28.933026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-40a75e7b-1266-4b59-920b-bc1c2fbc751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:28.999122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1167/2304] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:58:29.010949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:29.012754 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:29.104793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba0d0e-7f0a-4d1c-b265-bf83e6d46a0c admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:29.109436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ae9e0-9fdd-421c-a98b-a8e3b26fe143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:29.114502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ae9e0-9fdd-421c-a98b-a8e3b26fe143 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:29.115154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ae9e0-9fdd-421c-a98b-a8e3b26fe143 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:29.115821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ae9e0-9fdd-421c-a98b-a8e3b26fe143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:29.130057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ae9e0-9fdd-421c-a98b-a8e3b26fe143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:29.130307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ae9e0-9fdd-421c-a98b-a8e3b26fe143 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:29.141184 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1138/2305] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:58:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:58:29.573433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a75e7b-1266-4b59-920b-bc1c2fbc751e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:29.575253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-40a75e7b-1266-4b59-920b-bc1c2fbc751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:29.588249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a75e7b-1266-4b59-920b-bc1c2fbc751e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:29.596118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b37b9-9508-4717-80c5-db6c11aa8983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:29.596118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b37b9-9508-4717-80c5-db6c11aa8983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:29.596118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b37b9-9508-4717-80c5-db6c11aa8983 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:29.596118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b37b9-9508-4717-80c5-db6c11aa8983 admin admin] PATH_INFO: `/v3/projects/5219d434ae36439091135c08f5fa44de` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:29.597453 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b37b9-9508-4717-80c5-db6c11aa8983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5219d434ae36439091135c08f5fa44de)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:29.599667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b37b9-9508-4717-80c5-db6c11aa8983 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:29.770396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8d2b37b9-9508-4717-80c5-db6c11aa8983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5219d434ae36439091135c08f5fa44de', 'request_id': 'req-8d2b37b9-9508-4717-80c5-db6c11aa8983'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:29.777679 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8d2b37b9-9508-4717-80c5-db6c11aa8983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5219d434ae36439091135c08f5fa44de', 'request_id': 'req-8d2b37b9-9508-4717-80c5-db6c11aa8983'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:29.875773 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8d2b37b9-9508-4717-80c5-db6c11aa8983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5219d434ae36439091135c08f5fa44de', 'request_id': 'req-8d2b37b9-9508-4717-80c5-db6c11aa8983'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:29.889075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8d2b37b9-9508-4717-80c5-db6c11aa8983 admin admin] The token cache is being invalidate because project 5219d434ae36439091135c08f5fa44de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:58:29.889364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8d2b37b9-9508-4717-80c5-db6c11aa8983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:29.890110 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8d2b37b9-9508-4717-80c5-db6c11aa8983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:29.891700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1168/2306] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:58:29 2022] DELETE /identity/v3/projects/5219d434ae36439091135c08f5fa44de => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:30.125715 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ae9e0-9fdd-421c-a98b-a8e3b26fe143 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:30.127926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-073ae9e0-9fdd-421c-a98b-a8e3b26fe143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:30.143361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ae9e0-9fdd-421c-a98b-a8e3b26fe143 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:30.163582 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:30.209334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:30.210193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:30.213396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:30.213396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] PATH_INFO: `/v3/users/9742feebb05246d59d80bbb9e4f6a2ca` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:30.213396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9742feebb05246d59d80bbb9e4f6a2ca)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:30.226756 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:30.455262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9742feebb05246d59d80bbb9e4f6a2ca'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:30.560477 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9742feebb05246d59d80bbb9e4f6a2ca'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:30.561151 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9742feebb05246d59d80bbb9e4f6a2ca'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:30.561836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9742feebb05246d59d80bbb9e4f6a2ca'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:30.569273 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9742feebb05246d59d80bbb9e4f6a2ca'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:30.587183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1139/2307] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:58:30 2022] DELETE /identity/v3/users/9742feebb05246d59d80bbb9e4f6a2ca => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:30.592319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de0f6a-4ff3-40e0-aff9-f496bca05334 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:30.592812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de0f6a-4ff3-40e0-aff9-f496bca05334 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:30.593052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de0f6a-4ff3-40e0-aff9-f496bca05334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:30.614095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b4de0f6a-4ff3-40e0-aff9-f496bca05334 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:30.615381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b4de0f6a-4ff3-40e0-aff9-f496bca05334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:30.675963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1169/2308] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:30.686204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:30.687788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:30.755286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22847a9f-181f-4bce-8c88-72965ef20fb5 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:30.759124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c269c-fead-4a01-8cbb-8f97d77f750d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:30.760883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c269c-fead-4a01-8cbb-8f97d77f750d service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:30.761352 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c269c-fead-4a01-8cbb-8f97d77f750d service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:30.761629 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c269c-fead-4a01-8cbb-8f97d77f750d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:30.772712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c269c-fead-4a01-8cbb-8f97d77f750d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:30.774859 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c269c-fead-4a01-8cbb-8f97d77f750d service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:30.785652 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1140/2309] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:58:31.387856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de0f6a-4ff3-40e0-aff9-f496bca05334 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:31.389320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b4de0f6a-4ff3-40e0-aff9-f496bca05334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:31.411990 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de0f6a-4ff3-40e0-aff9-f496bca05334 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:31.426345 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2dadf-cac4-43a7-b1db-4054336f61e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:31.427137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2dadf-cac4-43a7-b1db-4054336f61e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:31.427342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2dadf-cac4-43a7-b1db-4054336f61e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:31.427596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2dadf-cac4-43a7-b1db-4054336f61e3 admin admin] PATH_INFO: `/v3/projects/f19f488a52f64b53b4b194364e1db3a9` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:31.428845 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2dadf-cac4-43a7-b1db-4054336f61e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f19f488a52f64b53b4b194364e1db3a9)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:31.430669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2dadf-cac4-43a7-b1db-4054336f61e3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:31.456579 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:31.574513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-48c2dadf-cac4-43a7-b1db-4054336f61e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f19f488a52f64b53b4b194364e1db3a9', 'request_id': 'req-48c2dadf-cac4-43a7-b1db-4054336f61e3'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:31.581437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-48c2dadf-cac4-43a7-b1db-4054336f61e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f19f488a52f64b53b4b194364e1db3a9', 'request_id': 'req-48c2dadf-cac4-43a7-b1db-4054336f61e3'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:31.674948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-48c2dadf-cac4-43a7-b1db-4054336f61e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f19f488a52f64b53b4b194364e1db3a9', 'request_id': 'req-48c2dadf-cac4-43a7-b1db-4054336f61e3'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:31.688210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-48c2dadf-cac4-43a7-b1db-4054336f61e3 admin admin] The token cache is being invalidate because project f19f488a52f64b53b4b194364e1db3a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:58:31.688611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-48c2dadf-cac4-43a7-b1db-4054336f61e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:31.689389 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-48c2dadf-cac4-43a7-b1db-4054336f61e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:31.691217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1170/2310] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:58:31 2022] DELETE /identity/v3/projects/f19f488a52f64b53b4b194364e1db3a9 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:31.696141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bc036-3c92-4bd2-86af-5f701a0828e9 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:31.696704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bc036-3c92-4bd2-86af-5f701a0828e9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:31.696821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bc036-3c92-4bd2-86af-5f701a0828e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:31.717348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-a14bc036-3c92-4bd2-86af-5f701a0828e9 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:31.718556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a14bc036-3c92-4bd2-86af-5f701a0828e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:31.762538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:31.780087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1141/2311] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:58:31.789103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f7587-9723-40dc-befd-bfa9f86d573a None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:31.789487 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f7587-9723-40dc-befd-bfa9f86d573a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:31.789772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f7587-9723-40dc-befd-bfa9f86d573a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:31.810339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-be5f7587-9723-40dc-befd-bfa9f86d573a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:31.811592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-be5f7587-9723-40dc-befd-bfa9f86d573a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:31.819970 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1171/2312] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:31.826515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14bc036-3c92-4bd2-86af-5f701a0828e9 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:31.838605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14bc036-3c92-4bd2-86af-5f701a0828e9 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:31.843892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aacded-078c-4571-8773-d037fa059f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:31.844577 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aacded-078c-4571-8773-d037fa059f1b admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:31.844925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aacded-078c-4571-8773-d037fa059f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:31.845261 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aacded-078c-4571-8773-d037fa059f1b admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:31.846031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aacded-078c-4571-8773-d037fa059f1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:58:31.846397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aacded-078c-4571-8773-d037fa059f1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:31.848180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aacded-078c-4571-8773-d037fa059f1b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:31.866806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1142/2313] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:58:31.871107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f7587-9723-40dc-befd-bfa9f86d573a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:31.880935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f7587-9723-40dc-befd-bfa9f86d573a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:31.886816 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26432d-cfbc-42a4-9e7e-909196cd4fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:31.887436 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26432d-cfbc-42a4-9e7e-909196cd4fab admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:31.887718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26432d-cfbc-42a4-9e7e-909196cd4fab admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:31.887994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26432d-cfbc-42a4-9e7e-909196cd4fab admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:31.894870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26432d-cfbc-42a4-9e7e-909196cd4fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:31.894870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26432d-cfbc-42a4-9e7e-909196cd4fab admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:31.986054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1172/2314] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:58:31 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:58:31.987257 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aacded-078c-4571-8773-d037fa059f1b admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:31.997763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aacded-078c-4571-8773-d037fa059f1b admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:32.069615 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9290d-a8d1-44a1-9668-a313a0423726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:32.071830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9290d-a8d1-44a1-9668-a313a0423726 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:32.073156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9290d-a8d1-44a1-9668-a313a0423726 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:32.073849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9290d-a8d1-44a1-9668-a313a0423726 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:32.075678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9290d-a8d1-44a1-9668-a313a0423726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:32.090541 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9290d-a8d1-44a1-9668-a313a0423726 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:32.172884 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1143/2315] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:58:31 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:58:32.177432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26432d-cfbc-42a4-9e7e-909196cd4fab admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:32.188211 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26432d-cfbc-42a4-9e7e-909196cd4fab admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:32.192779 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4733b-a48c-4f60-bb17-2aa82941cff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:32.193382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4733b-a48c-4f60-bb17-2aa82941cff4 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:32.193717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4733b-a48c-4f60-bb17-2aa82941cff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:32.193932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4733b-a48c-4f60-bb17-2aa82941cff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:32.194604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4733b-a48c-4f60-bb17-2aa82941cff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:32.196197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4733b-a48c-4f60-bb17-2aa82941cff4 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:32.210552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1173/2316] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:58:32.212244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9290d-a8d1-44a1-9668-a313a0423726 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:32.223094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9290d-a8d1-44a1-9668-a313a0423726 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:32.232785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed4908-3290-4967-b960-adb4f7df0eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:32.232785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed4908-3290-4967-b960-adb4f7df0eba admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:32.232785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed4908-3290-4967-b960-adb4f7df0eba admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:32.232785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed4908-3290-4967-b960-adb4f7df0eba admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:32.233680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed4908-3290-4967-b960-adb4f7df0eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:32.233680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed4908-3290-4967-b960-adb4f7df0eba admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:32.236786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59542b0f-8fd3-4be9-919b-5984709880b8 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:32.237067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59542b0f-8fd3-4be9-919b-5984709880b8 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:32.237338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59542b0f-8fd3-4be9-919b-5984709880b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:32.245367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1144/2317] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:58:32.254134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed4908-3290-4967-b960-adb4f7df0eba admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:32.257268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-59542b0f-8fd3-4be9-919b-5984709880b8 None None] MFA Rules not processed for user `63a2902f465f4fd1aed80060c2669259`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:32.258426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-59542b0f-8fd3-4be9-919b-5984709880b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:32.264944 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed4908-3290-4967-b960-adb4f7df0eba admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:32.269860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef5bd1-f8f5-40a7-a645-7958437868c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:32.270547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef5bd1-f8f5-40a7-a645-7958437868c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:32.270834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef5bd1-f8f5-40a7-a645-7958437868c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:32.271111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef5bd1-f8f5-40a7-a645-7958437868c1 admin admin] PATH_INFO: `/v3/projects/d427be7da6084e009b87ff500858cdc8/users/4c96322c48c146c7b8bcb9959f533820/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:32.291516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef5bd1-f8f5-40a7-a645-7958437868c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d427be7da6084e009b87ff500858cdc8, user_id=4c96322c48c146c7b8bcb9959f533820, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:32.291516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef5bd1-f8f5-40a7-a645-7958437868c1 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:32.317118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1174/2318] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:32.377054 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1145/2319] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:58:32 2022] PUT /identity/v3/projects/d427be7da6084e009b87ff500858cdc8/users/4c96322c48c146c7b8bcb9959f533820/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:32.382102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048c7e9-802d-4535-b887-f67b196d7a0f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:32.382393 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048c7e9-802d-4535-b887-f67b196d7a0f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:32.382702 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048c7e9-802d-4535-b887-f67b196d7a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:32.403415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-6048c7e9-802d-4535-b887-f67b196d7a0f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:32.404625 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6048c7e9-802d-4535-b887-f67b196d7a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:32.467709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1175/2320] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:32.477900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef5bd1-f8f5-40a7-a645-7958437868c1 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:32.479989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-51ef5bd1-f8f5-40a7-a645-7958437868c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:32.539691 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef5bd1-f8f5-40a7-a645-7958437868c1 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:32.543788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01e266-ed1d-4d87-95e5-9453bf4e535a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:32.544366 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01e266-ed1d-4d87-95e5-9453bf4e535a service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:32.544644 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01e266-ed1d-4d87-95e5-9453bf4e535a service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:32.544916 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01e266-ed1d-4d87-95e5-9453bf4e535a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:32.554402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01e266-ed1d-4d87-95e5-9453bf4e535a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:32.555891 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01e266-ed1d-4d87-95e5-9453bf4e535a service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:32.566536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1146/2321] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:58:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:58:33.139186 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048c7e9-802d-4535-b887-f67b196d7a0f None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:33.140967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6048c7e9-802d-4535-b887-f67b196d7a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:33.155739 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048c7e9-802d-4535-b887-f67b196d7a0f None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:33.160548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:33.161174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:33.161446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:33.161714 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] PATH_INFO: `/v3/users/2879dddbb34842feb8f01d98fe9c0ec9` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:33.162852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2879dddbb34842feb8f01d98fe9c0ec9)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:33.164790 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:33.171514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:33.406468 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2879dddbb34842feb8f01d98fe9c0ec9'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:33.484353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2879dddbb34842feb8f01d98fe9c0ec9'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:33.485082 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2879dddbb34842feb8f01d98fe9c0ec9'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:33.485702 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2879dddbb34842feb8f01d98fe9c0ec9'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:33.499181 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2879dddbb34842feb8f01d98fe9c0ec9'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:33.515710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1176/2322] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:58:33 2022] DELETE /identity/v3/users/2879dddbb34842feb8f01d98fe9c0ec9 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:33.518612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e118c3d-0aa1-4c24-884a-875efef4d937 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:33.519196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e118c3d-0aa1-4c24-884a-875efef4d937 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:33.519196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e118c3d-0aa1-4c24-884a-875efef4d937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:33.541577 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-9e118c3d-0aa1-4c24-884a-875efef4d937 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:33.542994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9e118c3d-0aa1-4c24-884a-875efef4d937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:33.573569 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:33.612473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1147/2323] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:33.622601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:33.623999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:33.686509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cef38c-0cc8-40ae-8283-4f85d0bcae7f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:33.690942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d04089-8ef7-480a-9a78-7357806a84f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:33.691599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d04089-8ef7-480a-9a78-7357806a84f2 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:33.691879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d04089-8ef7-480a-9a78-7357806a84f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:33.700360 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d04089-8ef7-480a-9a78-7357806a84f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:33.706785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d04089-8ef7-480a-9a78-7357806a84f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:33.708238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d04089-8ef7-480a-9a78-7357806a84f2 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:33.719787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1177/2324] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:58:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:58:34.221143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e118c3d-0aa1-4c24-884a-875efef4d937 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:34.222852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9e118c3d-0aa1-4c24-884a-875efef4d937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:34.243041 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e118c3d-0aa1-4c24-884a-875efef4d937 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:34.248819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:34.249522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:34.249804 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:34.250349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] PATH_INFO: `/v3/projects/deb5ae2fa5db48949e56181fbc70f09a` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:34.252137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deb5ae2fa5db48949e56181fbc70f09a)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:34.253943 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:34.404515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deb5ae2fa5db48949e56181fbc70f09a', 'request_id': 'req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:34.412598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deb5ae2fa5db48949e56181fbc70f09a', 'request_id': 'req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:34.532459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deb5ae2fa5db48949e56181fbc70f09a', 'request_id': 'req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:34.549121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] The token cache is being invalidate because project deb5ae2fa5db48949e56181fbc70f09a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:58:34.549408 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:34.550288 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:34.552027 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1148/2325] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:58:34 2022] DELETE /identity/v3/projects/deb5ae2fa5db48949e56181fbc70f09a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:37.716759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b84e7a-467e-4a43-8b8b-1466a985de56 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:37.717311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b84e7a-467e-4a43-8b8b-1466a985de56 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:37.717311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b84e7a-467e-4a43-8b8b-1466a985de56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:37.754922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-37b84e7a-467e-4a43-8b8b-1466a985de56 None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:37.756162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-37b84e7a-467e-4a43-8b8b-1466a985de56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:37.894417 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1178/2326] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 12:58:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:37.953683 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:37.957420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:38.084088 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c8fbc-a674-44cc-a012-79fc2ff1e9f7 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:38.090777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40aaf1-0ca5-49ce-83bf-55199321fd29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:38.091666 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40aaf1-0ca5-49ce-83bf-55199321fd29 service swift] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:38.092008 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40aaf1-0ca5-49ce-83bf-55199321fd29 service swift] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:38.092377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40aaf1-0ca5-49ce-83bf-55199321fd29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:38.108031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40aaf1-0ca5-49ce-83bf-55199321fd29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:38.110216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40aaf1-0ca5-49ce-83bf-55199321fd29 service swift] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:38.125783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1149/2327] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 12:58:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:58:41.921954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10461b-7f05-4a16-b0a4-c60f08a60803 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:41.922622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10461b-7f05-4a16-b0a4-c60f08a60803 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:41.922622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10461b-7f05-4a16-b0a4-c60f08a60803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:41.957607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-2b10461b-7f05-4a16-b0a4-c60f08a60803 None None] MFA Rules not processed for user `4c96322c48c146c7b8bcb9959f533820`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:41.962031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2b10461b-7f05-4a16-b0a4-c60f08a60803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:42.018892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:42.070814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1179/2328] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:42.075535 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40aaf1-0ca5-49ce-83bf-55199321fd29 service swift] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:42.077749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ba40aaf1-0ca5-49ce-83bf-55199321fd29 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:42.208793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:42.243444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40aaf1-0ca5-49ce-83bf-55199321fd29 service swift] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:42.250820 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c36a7b-ebe3-4814-bd76-36bf5f0b129a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:42.251520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c36a7b-ebe3-4814-bd76-36bf5f0b129a admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:42.251795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c36a7b-ebe3-4814-bd76-36bf5f0b129a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:42.252069 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c36a7b-ebe3-4814-bd76-36bf5f0b129a admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:42.252869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c36a7b-ebe3-4814-bd76-36bf5f0b129a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:42.254824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c36a7b-ebe3-4814-bd76-36bf5f0b129a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:42.504619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1150/2329] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:58:42 2022] POST /identity/v3/projects => generated 375 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:58:42.508240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10461b-7f05-4a16-b0a4-c60f08a60803 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:42.534964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10461b-7f05-4a16-b0a4-c60f08a60803 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:42.668045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc79395-b92c-452b-a01e-35d36967aea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:42.669894 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc79395-b92c-452b-a01e-35d36967aea7 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:42.670976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc79395-b92c-452b-a01e-35d36967aea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:42.671665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc79395-b92c-452b-a01e-35d36967aea7 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:42.672913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc79395-b92c-452b-a01e-35d36967aea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:42.676654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc79395-b92c-452b-a01e-35d36967aea7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:42.803000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1180/2330] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:58:42 2022] POST /identity/v3/users => generated 294 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:58:42.804525 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c36a7b-ebe3-4814-bd76-36bf5f0b129a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:42.817526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c36a7b-ebe3-4814-bd76-36bf5f0b129a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:42.824373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25248ab1-8369-48ed-bc3b-e33d6a0f40af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:42.825168 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25248ab1-8369-48ed-bc3b-e33d6a0f40af admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:42.825498 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25248ab1-8369-48ed-bc3b-e33d6a0f40af admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:42.825840 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25248ab1-8369-48ed-bc3b-e33d6a0f40af admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:42.826605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25248ab1-8369-48ed-bc3b-e33d6a0f40af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:42.831969 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25248ab1-8369-48ed-bc3b-e33d6a0f40af admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:42.846617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1151/2331] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:58:42.847932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc79395-b92c-452b-a01e-35d36967aea7 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:42.860125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc79395-b92c-452b-a01e-35d36967aea7 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:42.865481 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb8510-9582-4a86-95ef-fc0e4b86d713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:42.866254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb8510-9582-4a86-95ef-fc0e4b86d713 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:42.866672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb8510-9582-4a86-95ef-fc0e4b86d713 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:42.867057 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb8510-9582-4a86-95ef-fc0e4b86d713 admin admin] PATH_INFO: `/v3/projects/065af61a31cd48ddb0299de7aad02a15/users/fe7cff0184954f0d93e9aa9b5c346c75/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:42.888215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb8510-9582-4a86-95ef-fc0e4b86d713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=065af61a31cd48ddb0299de7aad02a15, user_id=fe7cff0184954f0d93e9aa9b5c346c75, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:42.891059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb8510-9582-4a86-95ef-fc0e4b86d713 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:42.983900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1181/2332] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:58:42 2022] PUT /identity/v3/projects/065af61a31cd48ddb0299de7aad02a15/users/fe7cff0184954f0d93e9aa9b5c346c75/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:42.988176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25248ab1-8369-48ed-bc3b-e33d6a0f40af admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:43.000790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25248ab1-8369-48ed-bc3b-e33d6a0f40af admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:43.147796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ddef9-c3e9-485f-9c75-547fb8adfe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:43.149084 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ddef9-c3e9-485f-9c75-547fb8adfe90 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:43.149487 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ddef9-c3e9-485f-9c75-547fb8adfe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:43.149942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ddef9-c3e9-485f-9c75-547fb8adfe90 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:43.151235 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ddef9-c3e9-485f-9c75-547fb8adfe90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:43.153522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ddef9-c3e9-485f-9c75-547fb8adfe90 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:43.167097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1152/2333] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:58:43.171033 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb8510-9582-4a86-95ef-fc0e4b86d713 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:43.188781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb8510-9582-4a86-95ef-fc0e4b86d713 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:43.194820 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e9b7c-0fea-4b42-bb7f-4a74b8a2ac98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:43.195505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e9b7c-0fea-4b42-bb7f-4a74b8a2ac98 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:43.198805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e9b7c-0fea-4b42-bb7f-4a74b8a2ac98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:43.198805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e9b7c-0fea-4b42-bb7f-4a74b8a2ac98 admin admin] PATH_INFO: `/v3/projects/065af61a31cd48ddb0299de7aad02a15/users/fe7cff0184954f0d93e9aa9b5c346c75/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:43.200048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e9b7c-0fea-4b42-bb7f-4a74b8a2ac98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=065af61a31cd48ddb0299de7aad02a15, user_id=fe7cff0184954f0d93e9aa9b5c346c75, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:43.202950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e9b7c-0fea-4b42-bb7f-4a74b8a2ac98 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:43.226641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1182/2334] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:58:43 2022] PUT /identity/v3/projects/065af61a31cd48ddb0299de7aad02a15/users/fe7cff0184954f0d93e9aa9b5c346c75/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:48.281834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ddef9-c3e9-485f-9c75-547fb8adfe90 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:48.283661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-661ddef9-c3e9-485f-9c75-547fb8adfe90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:48.304519 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ddef9-c3e9-485f-9c75-547fb8adfe90 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:48.395416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:48.396916 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:48.397644 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:48.397934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] PATH_INFO: `/v3/users/33a9a55dd16b49dbb97e6b25bcc1e438` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:48.399162 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a9a55dd16b49dbb97e6b25bcc1e438)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:48.402215 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:48.676471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a9a55dd16b49dbb97e6b25bcc1e438'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:48.752209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a9a55dd16b49dbb97e6b25bcc1e438'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:48.755997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a9a55dd16b49dbb97e6b25bcc1e438'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:48.756755 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a9a55dd16b49dbb97e6b25bcc1e438'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:48.759936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:48.772118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a9a55dd16b49dbb97e6b25bcc1e438'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:48.815213 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1153/2335] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:58:48 2022] DELETE /identity/v3/users/33a9a55dd16b49dbb97e6b25bcc1e438 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:48.819637 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bb383-093c-47f7-9c4d-24ab87dd8895 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:48.820166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bb383-093c-47f7-9c4d-24ab87dd8895 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:48.820166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bb383-093c-47f7-9c4d-24ab87dd8895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:48.855317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d16bb383-093c-47f7-9c4d-24ab87dd8895 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:48.856673 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d16bb383-093c-47f7-9c4d-24ab87dd8895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:48.958668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1183/2336] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:48.971101 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:48.975732 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:49.106997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206d1af-c536-441c-80e1-4b2996f2758b admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:49.121911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45f3d6-45fd-4b4e-a62a-590e1a5af84c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:49.121911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45f3d6-45fd-4b4e-a62a-590e1a5af84c service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:49.121911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45f3d6-45fd-4b4e-a62a-590e1a5af84c service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:49.121911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45f3d6-45fd-4b4e-a62a-590e1a5af84c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:49.145249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45f3d6-45fd-4b4e-a62a-590e1a5af84c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:49.147103 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45f3d6-45fd-4b4e-a62a-590e1a5af84c service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:49.168971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1154/2337] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:58:49.938032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bb383-093c-47f7-9c4d-24ab87dd8895 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:49.940928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d16bb383-093c-47f7-9c4d-24ab87dd8895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:49.975194 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bb383-093c-47f7-9c4d-24ab87dd8895 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:49.983194 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efaf90-3046-4597-b0ec-a46c23f2d4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:49.984691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efaf90-3046-4597-b0ec-a46c23f2d4a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:49.984691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efaf90-3046-4597-b0ec-a46c23f2d4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:49.984691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efaf90-3046-4597-b0ec-a46c23f2d4a0 admin admin] PATH_INFO: `/v3/projects/f0dcc96f9565435eabccde5ac25dbcd3` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:49.985662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efaf90-3046-4597-b0ec-a46c23f2d4a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0dcc96f9565435eabccde5ac25dbcd3)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:49.988376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efaf90-3046-4597-b0ec-a46c23f2d4a0 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:50.005499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:50.161488 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-69efaf90-3046-4597-b0ec-a46c23f2d4a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0dcc96f9565435eabccde5ac25dbcd3', 'request_id': 'req-69efaf90-3046-4597-b0ec-a46c23f2d4a0'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:50.174708 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-69efaf90-3046-4597-b0ec-a46c23f2d4a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0dcc96f9565435eabccde5ac25dbcd3', 'request_id': 'req-69efaf90-3046-4597-b0ec-a46c23f2d4a0'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:50.258168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-69efaf90-3046-4597-b0ec-a46c23f2d4a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0dcc96f9565435eabccde5ac25dbcd3', 'request_id': 'req-69efaf90-3046-4597-b0ec-a46c23f2d4a0'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:50.281094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-69efaf90-3046-4597-b0ec-a46c23f2d4a0 admin admin] The token cache is being invalidate because project f0dcc96f9565435eabccde5ac25dbcd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:58:50.281606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-69efaf90-3046-4597-b0ec-a46c23f2d4a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:50.283914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-69efaf90-3046-4597-b0ec-a46c23f2d4a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:58:50.287165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1184/2338] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:58:49 2022] DELETE /identity/v3/projects/f0dcc96f9565435eabccde5ac25dbcd3 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:58:50.292677 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d584d1-99c6-40a8-a68c-24decf33199f None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:50.293194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d584d1-99c6-40a8-a68c-24decf33199f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:50.293631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d584d1-99c6-40a8-a68c-24decf33199f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:50.339539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-d7d584d1-99c6-40a8-a68c-24decf33199f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:50.343543 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d7d584d1-99c6-40a8-a68c-24decf33199f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:50.484653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1155/2339] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:50.486418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276df44-6260-4631-929c-a80630ec7786 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:50.486727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276df44-6260-4631-929c-a80630ec7786 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:50.487004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276df44-6260-4631-929c-a80630ec7786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:50.513844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-8276df44-6260-4631-929c-a80630ec7786 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:50.515010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8276df44-6260-4631-929c-a80630ec7786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:50.524068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1185/2340] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:50.528898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d584d1-99c6-40a8-a68c-24decf33199f None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:50.540171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d584d1-99c6-40a8-a68c-24decf33199f None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:50.547902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44d7ec-34f2-49ab-bd2d-772154e8b663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:50.549340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44d7ec-34f2-49ab-bd2d-772154e8b663 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:50.549522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44d7ec-34f2-49ab-bd2d-772154e8b663 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:50.549896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44d7ec-34f2-49ab-bd2d-772154e8b663 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:50.550716 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44d7ec-34f2-49ab-bd2d-772154e8b663 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:58:50.551125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44d7ec-34f2-49ab-bd2d-772154e8b663 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:50.552879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44d7ec-34f2-49ab-bd2d-772154e8b663 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:50.580499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1156/2341] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:58:50.580799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276df44-6260-4631-929c-a80630ec7786 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:50.593610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276df44-6260-4631-929c-a80630ec7786 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:50.607156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ab7f3-0804-4411-856e-92c9f74c0df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:50.609619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ab7f3-0804-4411-856e-92c9f74c0df3 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:50.609914 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ab7f3-0804-4411-856e-92c9f74c0df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:50.610209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ab7f3-0804-4411-856e-92c9f74c0df3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:50.611931 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ab7f3-0804-4411-856e-92c9f74c0df3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:50.619199 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ab7f3-0804-4411-856e-92c9f74c0df3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:50.730145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1186/2342] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:58:50 2022] POST /identity/v3/projects => generated 399 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:58:50.731833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44d7ec-34f2-49ab-bd2d-772154e8b663 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:50.760511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44d7ec-34f2-49ab-bd2d-772154e8b663 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:50.845925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b4b9d-22f8-4375-bc47-2cee7e33fb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:50.846651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b4b9d-22f8-4375-bc47-2cee7e33fb04 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:50.846943 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b4b9d-22f8-4375-bc47-2cee7e33fb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:50.847217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b4b9d-22f8-4375-bc47-2cee7e33fb04 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:50.847867 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b4b9d-22f8-4375-bc47-2cee7e33fb04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:50.849873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b4b9d-22f8-4375-bc47-2cee7e33fb04 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:50.861363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:51.034790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1157/2343] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:58:50 2022] POST /identity/v3/users => generated 300 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:58:51.039131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ab7f3-0804-4411-856e-92c9f74c0df3 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:51.058715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ab7f3-0804-4411-856e-92c9f74c0df3 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:51.069079 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ab304-98a9-4e22-9997-62ba5d29f56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:51.069714 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ab304-98a9-4e22-9997-62ba5d29f56a admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:51.070911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ab304-98a9-4e22-9997-62ba5d29f56a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:51.071729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ab304-98a9-4e22-9997-62ba5d29f56a admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:51.072246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ab304-98a9-4e22-9997-62ba5d29f56a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:51.074112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ab304-98a9-4e22-9997-62ba5d29f56a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:51.089533 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1187/2344] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:58:51.093238 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b4b9d-22f8-4375-bc47-2cee7e33fb04 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:51.136265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b4b9d-22f8-4375-bc47-2cee7e33fb04 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:51.143793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312a6b4-17b1-4e39-908f-603e29ccddda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:51.144663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312a6b4-17b1-4e39-908f-603e29ccddda admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:51.145013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312a6b4-17b1-4e39-908f-603e29ccddda admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:51.145292 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312a6b4-17b1-4e39-908f-603e29ccddda admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:51.145937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312a6b4-17b1-4e39-908f-603e29ccddda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:51.156901 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312a6b4-17b1-4e39-908f-603e29ccddda admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:51.187545 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1158/2345] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:58:51.192514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ab304-98a9-4e22-9997-62ba5d29f56a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:51.211255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ab304-98a9-4e22-9997-62ba5d29f56a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:51.229540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442650b7-33d0-4a4c-9c08-e9292d70cbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:51.230773 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442650b7-33d0-4a4c-9c08-e9292d70cbd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:51.232764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442650b7-33d0-4a4c-9c08-e9292d70cbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:51.234046 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442650b7-33d0-4a4c-9c08-e9292d70cbd2 admin admin] PATH_INFO: `/v3/projects/581c77ea6e694a4b914b8fddce919b31/users/bb37fe5e26094de6a78ea1860888999c/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:51.276807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442650b7-33d0-4a4c-9c08-e9292d70cbd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=581c77ea6e694a4b914b8fddce919b31, user_id=bb37fe5e26094de6a78ea1860888999c, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:51.281117 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442650b7-33d0-4a4c-9c08-e9292d70cbd2 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:51.437577 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1188/2346] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:58:51 2022] PUT /identity/v3/projects/581c77ea6e694a4b914b8fddce919b31/users/bb37fe5e26094de6a78ea1860888999c/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:58:51.448283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c0ebb-6a85-4760-8a69-031aa6900247 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:51.448769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c0ebb-6a85-4760-8a69-031aa6900247 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:51.448882 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c0ebb-6a85-4760-8a69-031aa6900247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:51.511852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-c56c0ebb-6a85-4760-8a69-031aa6900247 None None] MFA Rules not processed for user `bb37fe5e26094de6a78ea1860888999c`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:51.513082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c56c0ebb-6a85-4760-8a69-031aa6900247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:51.665517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1159/2347] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:51 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:51.681617 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442650b7-33d0-4a4c-9c08-e9292d70cbd2 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:51.685267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-442650b7-33d0-4a4c-9c08-e9292d70cbd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:51.807699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442650b7-33d0-4a4c-9c08-e9292d70cbd2 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:51.816177 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30134c-5586-47d5-affe-3c1cb7f797f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:51.817099 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30134c-5586-47d5-affe-3c1cb7f797f6 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:51.817532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30134c-5586-47d5-affe-3c1cb7f797f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:51.817957 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30134c-5586-47d5-affe-3c1cb7f797f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:51.839334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30134c-5586-47d5-affe-3c1cb7f797f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:51.841067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30134c-5586-47d5-affe-3c1cb7f797f6 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:51.852454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1189/2348] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:58:51 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:58:52.977590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96829389-c397-4125-bc67-b45b23c688ef None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:52.977590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96829389-c397-4125-bc67-b45b23c688ef None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:52.977590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96829389-c397-4125-bc67-b45b23c688ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:53.013598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-96829389-c397-4125-bc67-b45b23c688ef None None] MFA Rules not processed for user `fe7cff0184954f0d93e9aa9b5c346c75`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:58:53.015098 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-96829389-c397-4125-bc67-b45b23c688ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:53.150581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1160/2349] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:58:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:58:53.162247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30134c-5586-47d5-affe-3c1cb7f797f6 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:58:53.163889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1f30134c-5586-47d5-affe-3c1cb7f797f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:53.218438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:58:53.258475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30134c-5586-47d5-affe-3c1cb7f797f6 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:58:53.274142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42eea5c-cba4-46fc-9198-aab606c4571d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:58:53.275773 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eea5c-cba4-46fc-9198-aab606c4571d service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:58:53.276174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eea5c-cba4-46fc-9198-aab606c4571d service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:58:53.276544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eea5c-cba4-46fc-9198-aab606c4571d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:58:53.278501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b42eea5c-cba4-46fc-9198-aab606c4571d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:58:53.439145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42eea5c-cba4-46fc-9198-aab606c4571d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:58:53.441952 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42eea5c-cba4-46fc-9198-aab606c4571d service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:58:53.460327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1190/2350] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:58:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:59:01.350074 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96829389-c397-4125-bc67-b45b23c688ef None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:01.409303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:59:01.415523 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96829389-c397-4125-bc67-b45b23c688ef None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:01.420066 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:01.420690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:01.420967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:01.421266 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] PATH_INFO: `/v3/users/bb37fe5e26094de6a78ea1860888999c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:01.422358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb37fe5e26094de6a78ea1860888999c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:01.424446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:01.662429 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb37fe5e26094de6a78ea1860888999c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:59:01.737850 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb37fe5e26094de6a78ea1860888999c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:59:01.738563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb37fe5e26094de6a78ea1860888999c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:59:01.739222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb37fe5e26094de6a78ea1860888999c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:59:01.748382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb37fe5e26094de6a78ea1860888999c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:59:01.771959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1161/2351] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 12:59:01 2022] DELETE /identity/v3/users/bb37fe5e26094de6a78ea1860888999c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:59:01.774927 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b263ff5-46cb-4bb8-8f94-52eee7da1a5a None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:01.775596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b263ff5-46cb-4bb8-8f94-52eee7da1a5a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:01.775596 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b263ff5-46cb-4bb8-8f94-52eee7da1a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:01.795799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-6b263ff5-46cb-4bb8-8f94-52eee7da1a5a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:01.797023 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6b263ff5-46cb-4bb8-8f94-52eee7da1a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:01.890879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1191/2352] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:59:01.903442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:01.909740 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:01.983008 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d68834-0985-4a43-8f83-48d54b5e23aa admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:01.987041 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d609de-7c86-463f-bcfa-bdfd0b3de410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:01.987806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d609de-7c86-463f-bcfa-bdfd0b3de410 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:01.988214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d609de-7c86-463f-bcfa-bdfd0b3de410 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:01.988559 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d609de-7c86-463f-bcfa-bdfd0b3de410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:01.999303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d609de-7c86-463f-bcfa-bdfd0b3de410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:02.000837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d609de-7c86-463f-bcfa-bdfd0b3de410 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:02.012514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1162/2353] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:59:02.473922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b263ff5-46cb-4bb8-8f94-52eee7da1a5a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:02.477884 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6b263ff5-46cb-4bb8-8f94-52eee7da1a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:02.495068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b263ff5-46cb-4bb8-8f94-52eee7da1a5a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:02.499854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae3b1b-2259-44f3-8484-b43df4f09b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:02.500680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae3b1b-2259-44f3-8484-b43df4f09b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:02.500818 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae3b1b-2259-44f3-8484-b43df4f09b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:02.501052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae3b1b-2259-44f3-8484-b43df4f09b83 admin admin] PATH_INFO: `/v3/projects/581c77ea6e694a4b914b8fddce919b31` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:02.502309 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae3b1b-2259-44f3-8484-b43df4f09b83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=581c77ea6e694a4b914b8fddce919b31)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:02.504262 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae3b1b-2259-44f3-8484-b43df4f09b83 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:02.633472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-96ae3b1b-2259-44f3-8484-b43df4f09b83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '581c77ea6e694a4b914b8fddce919b31', 'request_id': 'req-96ae3b1b-2259-44f3-8484-b43df4f09b83'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:59:02.640408 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-96ae3b1b-2259-44f3-8484-b43df4f09b83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '581c77ea6e694a4b914b8fddce919b31', 'request_id': 'req-96ae3b1b-2259-44f3-8484-b43df4f09b83'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:59:02.725974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-96ae3b1b-2259-44f3-8484-b43df4f09b83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '581c77ea6e694a4b914b8fddce919b31', 'request_id': 'req-96ae3b1b-2259-44f3-8484-b43df4f09b83'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:59:02.739681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:59:02.745646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-96ae3b1b-2259-44f3-8484-b43df4f09b83 admin admin] The token cache is being invalidate because project 581c77ea6e694a4b914b8fddce919b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:59:02.746980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-96ae3b1b-2259-44f3-8484-b43df4f09b83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:59:02.748024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-96ae3b1b-2259-44f3-8484-b43df4f09b83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:59:02.749726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1192/2354] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 12:59:02 2022] DELETE /identity/v3/projects/581c77ea6e694a4b914b8fddce919b31 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:59:02.754938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357428c7-67c9-462e-a2b4-07a3384eee49 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:02.755424 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357428c7-67c9-462e-a2b4-07a3384eee49 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:02.756039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357428c7-67c9-462e-a2b4-07a3384eee49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:02.778616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-357428c7-67c9-462e-a2b4-07a3384eee49 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:02.780415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-357428c7-67c9-462e-a2b4-07a3384eee49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:02.789941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771dc83-91b7-4b8e-b03c-156b1950bd97 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:02.792050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771dc83-91b7-4b8e-b03c-156b1950bd97 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:02.792328 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771dc83-91b7-4b8e-b03c-156b1950bd97 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:02.793713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1193/2355] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 12:59:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:59:02.800229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbf01c-ac3c-441a-8ec5-9a1e4b7e46fb None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:02.800502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbf01c-ac3c-441a-8ec5-9a1e4b7e46fb None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:02.800942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbf01c-ac3c-441a-8ec5-9a1e4b7e46fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:02.829649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0dbbf01c-ac3c-441a-8ec5-9a1e4b7e46fb None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:02.830940 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0dbbf01c-ac3c-441a-8ec5-9a1e4b7e46fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:02.872519 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1163/2356] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:59:02.882284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f43c3-2e00-465e-a383-880ac0d26f43 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:02.882621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f43c3-2e00-465e-a383-880ac0d26f43 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:02.882904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f43c3-2e00-465e-a383-880ac0d26f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:02.903506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-0b7f43c3-2e00-465e-a383-880ac0d26f43 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:02.904700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0b7f43c3-2e00-465e-a383-880ac0d26f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:02.907144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1194/2357] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 12:59:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:59:02.915968 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1164/2358] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:59:02.919931 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f43c3-2e00-465e-a383-880ac0d26f43 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:02.920412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbf01c-ac3c-441a-8ec5-9a1e4b7e46fb None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:02.922140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0dbbf01c-ac3c-441a-8ec5-9a1e4b7e46fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:02.932827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f43c3-2e00-465e-a383-880ac0d26f43 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:02.937808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1814149-21d0-4343-8c63-9b83f7e6380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:02.938512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1814149-21d0-4343-8c63-9b83f7e6380f admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:02.938862 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1814149-21d0-4343-8c63-9b83f7e6380f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:02.939201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1814149-21d0-4343-8c63-9b83f7e6380f admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:02.939912 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1814149-21d0-4343-8c63-9b83f7e6380f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:59:02.940280 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1814149-21d0-4343-8c63-9b83f7e6380f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:02.941939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1814149-21d0-4343-8c63-9b83f7e6380f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:02.950058 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:59:02.966255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1165/2359] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 12:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:59:02.966255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1814149-21d0-4343-8c63-9b83f7e6380f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:02.970886 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1814149-21d0-4343-8c63-9b83f7e6380f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:02.976149 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1c3f8-2f78-4e21-bbaf-48af0d36d274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:02.976738 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1c3f8-2f78-4e21-bbaf-48af0d36d274 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:02.977016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1c3f8-2f78-4e21-bbaf-48af0d36d274 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:02.977284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1c3f8-2f78-4e21-bbaf-48af0d36d274 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:02.978199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1c3f8-2f78-4e21-bbaf-48af0d36d274 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:02.980854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1c3f8-2f78-4e21-bbaf-48af0d36d274 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:02.983200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbf01c-ac3c-441a-8ec5-9a1e4b7e46fb None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:02.987184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1523e9-5dd1-4cba-a5eb-376e177a8dd0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff8fcb7fa2740bbb801badf800d2d6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:02.987880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1523e9-5dd1-4cba-a5eb-376e177a8dd0 service glance] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:02.988232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1523e9-5dd1-4cba-a5eb-376e177a8dd0 service glance] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:02.989061 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1523e9-5dd1-4cba-a5eb-376e177a8dd0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:03.000288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1523e9-5dd1-4cba-a5eb-376e177a8dd0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:03.001711 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1523e9-5dd1-4cba-a5eb-376e177a8dd0 service glance] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:03.012975 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1195/2360] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:59:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:59:03.102943 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1166/2361] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:59:02 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:59:03.107597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1523e9-5dd1-4cba-a5eb-376e177a8dd0 service glance] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:03.122053 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1523e9-5dd1-4cba-a5eb-376e177a8dd0 service glance] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:03.185696 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f44e3-0b09-46a6-a778-d018490e23cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:03.190280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f44e3-0b09-46a6-a778-d018490e23cb admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:03.190280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f44e3-0b09-46a6-a778-d018490e23cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:03.190280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f44e3-0b09-46a6-a778-d018490e23cb admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:03.190280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f44e3-0b09-46a6-a778-d018490e23cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:03.190280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f44e3-0b09-46a6-a778-d018490e23cb admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:03.331134 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1196/2362] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:59:03 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:59:03.335402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1c3f8-2f78-4e21-bbaf-48af0d36d274 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:03.344865 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1ccbe-5bae-4bd6-a4bf-697980defa0f None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:03.345227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1ccbe-5bae-4bd6-a4bf-697980defa0f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:03.345703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1ccbe-5bae-4bd6-a4bf-697980defa0f None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:03.347932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1197/2363] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 12:59:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:59:03.348059 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1c3f8-2f78-4e21-bbaf-48af0d36d274 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:03.355561 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe982ac-e4eb-4cb6-8ddb-3547662acf0d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:03.355561 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe982ac-e4eb-4cb6-8ddb-3547662acf0d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:03.359379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe982ac-e4eb-4cb6-8ddb-3547662acf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:03.362006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376f749-3647-429c-b50c-bec161a1bdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:03.362731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376f749-3647-429c-b50c-bec161a1bdf6 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:03.363012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376f749-3647-429c-b50c-bec161a1bdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:03.363284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376f749-3647-429c-b50c-bec161a1bdf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:03.364206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376f749-3647-429c-b50c-bec161a1bdf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:03.365883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376f749-3647-429c-b50c-bec161a1bdf6 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:03.378009 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1167/2364] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:59:03.380360 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-4fe982ac-e4eb-4cb6-8ddb-3547662acf0d None None] MFA Rules not processed for user `e46bd9d49bf64013ae58770b0284d81e`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:03.381648 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4fe982ac-e4eb-4cb6-8ddb-3547662acf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:03.382000 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376f749-3647-429c-b50c-bec161a1bdf6 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:03.395663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376f749-3647-429c-b50c-bec161a1bdf6 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:03.401478 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ae940-bfac-4681-8ad9-9209546e0e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:03.402077 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ae940-bfac-4681-8ad9-9209546e0e61 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:03.402361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ae940-bfac-4681-8ad9-9209546e0e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:03.402669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ae940-bfac-4681-8ad9-9209546e0e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:03.403314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ae940-bfac-4681-8ad9-9209546e0e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:03.405224 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:59:03.405388 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ae940-bfac-4681-8ad9-9209546e0e61 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:03.416422 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1168/2365] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:59:03.422678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ae940-bfac-4681-8ad9-9209546e0e61 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:03.433465 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ae940-bfac-4681-8ad9-9209546e0e61 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:03.441803 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfdd19-a807-4fbf-afbb-889a2af09625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:03.442811 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfdd19-a807-4fbf-afbb-889a2af09625 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:03.443164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfdd19-a807-4fbf-afbb-889a2af09625 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:03.443506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfdd19-a807-4fbf-afbb-889a2af09625 admin admin] PATH_INFO: `/v3/projects/bfb3240493c84bda8102619400759342/users/eb144bc11f4b4cd59e1f70ebd6b7207c/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:03.450751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1198/2366] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 12:59:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:59:03.463757 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe982ac-e4eb-4cb6-8ddb-3547662acf0d None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:03.465275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfdd19-a807-4fbf-afbb-889a2af09625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb3240493c84bda8102619400759342, user_id=eb144bc11f4b4cd59e1f70ebd6b7207c, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:03.465478 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4fe982ac-e4eb-4cb6-8ddb-3547662acf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:03.468195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfdd19-a807-4fbf-afbb-889a2af09625 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:03.478286 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe982ac-e4eb-4cb6-8ddb-3547662acf0d None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:03.482476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0766948-e8ee-47db-9c3f-1950249c97d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:03.483104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0766948-e8ee-47db-9c3f-1950249c97d9 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:03.483383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0766948-e8ee-47db-9c3f-1950249c97d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:03.483656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0766948-e8ee-47db-9c3f-1950249c97d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:03.490663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0766948-e8ee-47db-9c3f-1950249c97d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:03.497833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0766948-e8ee-47db-9c3f-1950249c97d9 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:03.499996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1199/2367] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:59:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:59:03.581425 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1169/2368] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:59:03 2022] PUT /identity/v3/projects/bfb3240493c84bda8102619400759342/users/eb144bc11f4b4cd59e1f70ebd6b7207c/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:59:03.585778 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23906c00-3f83-4ad1-b0db-bb60c5d4fc53 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:03.586263 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23906c00-3f83-4ad1-b0db-bb60c5d4fc53 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:03.586550 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c102dc-82dd-478c-b47e-d1dee64e60e8 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:03.586665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23906c00-3f83-4ad1-b0db-bb60c5d4fc53 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:03.586915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c102dc-82dd-478c-b47e-d1dee64e60e8 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:03.587758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c102dc-82dd-478c-b47e-d1dee64e60e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:03.589051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1200/2369] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 12:59:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:59:03.594285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756f323-853a-42ed-a718-8f46098c6104 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:03.595479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756f323-853a-42ed-a718-8f46098c6104 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:03.595479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756f323-853a-42ed-a718-8f46098c6104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:03.608394 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-77c102dc-82dd-478c-b47e-d1dee64e60e8 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:03.609629 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-77c102dc-82dd-478c-b47e-d1dee64e60e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:03.615502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d756f323-853a-42ed-a718-8f46098c6104 None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:03.616671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d756f323-853a-42ed-a718-8f46098c6104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:03.673634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1170/2370] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:59:03.687376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c102dc-82dd-478c-b47e-d1dee64e60e8 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:03.692892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1201/2371] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 12:59:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:59:03.711316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756f323-853a-42ed-a718-8f46098c6104 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:03.715842 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d756f323-853a-42ed-a718-8f46098c6104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:03.766418 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c102dc-82dd-478c-b47e-d1dee64e60e8 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:03.770634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278092ee-76f8-4b3f-a453-4d39c7fdb4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:03.773548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278092ee-76f8-4b3f-a453-4d39c7fdb4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:03.773548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278092ee-76f8-4b3f-a453-4d39c7fdb4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:03.773548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278092ee-76f8-4b3f-a453-4d39c7fdb4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:03.782239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278092ee-76f8-4b3f-a453-4d39c7fdb4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:03.783873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278092ee-76f8-4b3f-a453-4d39c7fdb4a0 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:03.796272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1171/2372] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:59:03.798802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756f323-853a-42ed-a718-8f46098c6104 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:03.803150 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f030af-9c2f-4aea-a610-8cac9296203f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:03.813911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f030af-9c2f-4aea-a610-8cac9296203f service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:03.813911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f030af-9c2f-4aea-a610-8cac9296203f service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:03.813911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f030af-9c2f-4aea-a610-8cac9296203f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:03.816395 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f030af-9c2f-4aea-a610-8cac9296203f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:03.817932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f030af-9c2f-4aea-a610-8cac9296203f service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:03.831053 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1202/2373] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:59:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:59:08.833619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b236966-8c08-4104-8ed2-baf03578512c None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:08.834692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b236966-8c08-4104-8ed2-baf03578512c None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:08.835127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b236966-8c08-4104-8ed2-baf03578512c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:08.874092 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-6b236966-8c08-4104-8ed2-baf03578512c None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:08.874830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6b236966-8c08-4104-8ed2-baf03578512c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:08.922209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:59:08.969594 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1172/2374] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 12:59:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:59:08.984917 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f030af-9c2f-4aea-a610-8cac9296203f service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:08.986981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e9f030af-9c2f-4aea-a610-8cac9296203f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:09.132721 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:59:09.181563 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f030af-9c2f-4aea-a610-8cac9296203f service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:09.197226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92c356-a196-4d3e-a201-d5219cc453e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:09.199438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92c356-a196-4d3e-a201-d5219cc453e0 service swift] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:09.199438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92c356-a196-4d3e-a201-d5219cc453e0 service swift] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:09.199438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92c356-a196-4d3e-a201-d5219cc453e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:09.221309 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92c356-a196-4d3e-a201-d5219cc453e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:09.225273 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92c356-a196-4d3e-a201-d5219cc453e0 service swift] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:09.235249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1203/2375] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 12:59:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:59:10.275287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b017383-6bf6-4526-aae2-b75b3811d0f5 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:10.276690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b017383-6bf6-4526-aae2-b75b3811d0f5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:10.280518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b017383-6bf6-4526-aae2-b75b3811d0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:10.341541 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-0b017383-6bf6-4526-aae2-b75b3811d0f5 None None] MFA Rules not processed for user `eb144bc11f4b4cd59e1f70ebd6b7207c`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:10.347085 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0b017383-6bf6-4526-aae2-b75b3811d0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:10.461928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1173/2376] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:59:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:59:10.463607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92c356-a196-4d3e-a201-d5219cc453e0 service swift] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:10.492353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92c356-a196-4d3e-a201-d5219cc453e0 service swift] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:10.501351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60474a-3b35-4188-b01e-486564f072a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:10.502723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60474a-3b35-4188-b01e-486564f072a4 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:10.503531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60474a-3b35-4188-b01e-486564f072a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:10.504189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60474a-3b35-4188-b01e-486564f072a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:10.507595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e60474a-3b35-4188-b01e-486564f072a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:10.511438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e60474a-3b35-4188-b01e-486564f072a4 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:10.634942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1204/2377] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 12:59:10 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:59:10.639373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b017383-6bf6-4526-aae2-b75b3811d0f5 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:10.658650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b017383-6bf6-4526-aae2-b75b3811d0f5 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:10.772732 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a843f97-f641-4116-afd2-3a8b4550ec7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:10.773596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a843f97-f641-4116-afd2-3a8b4550ec7f admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:10.773928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a843f97-f641-4116-afd2-3a8b4550ec7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:10.774685 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a843f97-f641-4116-afd2-3a8b4550ec7f admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:10.775455 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a843f97-f641-4116-afd2-3a8b4550ec7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:10.777511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a843f97-f641-4116-afd2-3a8b4550ec7f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:10.869731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1174/2378] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 12:59:10 2022] POST /identity/v3/users => generated 299 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:59:10.873842 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60474a-3b35-4188-b01e-486564f072a4 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:10.892109 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60474a-3b35-4188-b01e-486564f072a4 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:10.907535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ad1b1-468f-43be-8b44-aa0afe991fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:10.908170 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ad1b1-468f-43be-8b44-aa0afe991fca admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:10.908452 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ad1b1-468f-43be-8b44-aa0afe991fca admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:10.908748 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ad1b1-468f-43be-8b44-aa0afe991fca admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:10.909378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ad1b1-468f-43be-8b44-aa0afe991fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:10.911200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ad1b1-468f-43be-8b44-aa0afe991fca admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:10.941105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1205/2379] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:59:10.942449 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a843f97-f641-4116-afd2-3a8b4550ec7f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:10.956579 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a843f97-f641-4116-afd2-3a8b4550ec7f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:10.961444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deafd60-acfb-4656-8a1f-3f8cfe5f2b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:10.962716 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deafd60-acfb-4656-8a1f-3f8cfe5f2b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:10.963005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deafd60-acfb-4656-8a1f-3f8cfe5f2b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:10.963281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deafd60-acfb-4656-8a1f-3f8cfe5f2b99 admin admin] PATH_INFO: `/v3/projects/a4980c26d9b44dadbaa680d7ac6994d3/users/af4c6cd209c149ebb2d32fff17972715/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:10.979970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deafd60-acfb-4656-8a1f-3f8cfe5f2b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4980c26d9b44dadbaa680d7ac6994d3, user_id=af4c6cd209c149ebb2d32fff17972715, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:10.983075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deafd60-acfb-4656-8a1f-3f8cfe5f2b99 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:11.089647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1175/2380] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:59:10 2022] PUT /identity/v3/projects/a4980c26d9b44dadbaa680d7ac6994d3/users/af4c6cd209c149ebb2d32fff17972715/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:59:11.093785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ad1b1-468f-43be-8b44-aa0afe991fca admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:11.114383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ad1b1-468f-43be-8b44-aa0afe991fca admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:11.229227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779aee6c-8237-448f-b6a8-a9f94ccb22f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:11.230115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779aee6c-8237-448f-b6a8-a9f94ccb22f1 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:11.230416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779aee6c-8237-448f-b6a8-a9f94ccb22f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:11.230733 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779aee6c-8237-448f-b6a8-a9f94ccb22f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:11.231534 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779aee6c-8237-448f-b6a8-a9f94ccb22f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:11.233131 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779aee6c-8237-448f-b6a8-a9f94ccb22f1 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:11.246038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1206/2381] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 12:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:59:11.250031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deafd60-acfb-4656-8a1f-3f8cfe5f2b99 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:11.263247 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deafd60-acfb-4656-8a1f-3f8cfe5f2b99 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:11.275503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014d40b-2251-4464-ab00-b276c7559302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:11.276208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014d40b-2251-4464-ab00-b276c7559302 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:11.276463 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014d40b-2251-4464-ab00-b276c7559302 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:11.276742 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014d40b-2251-4464-ab00-b276c7559302 admin admin] PATH_INFO: `/v3/projects/a4980c26d9b44dadbaa680d7ac6994d3/users/af4c6cd209c149ebb2d32fff17972715/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:11.280921 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014d40b-2251-4464-ab00-b276c7559302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4980c26d9b44dadbaa680d7ac6994d3, user_id=af4c6cd209c149ebb2d32fff17972715, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:11.283622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014d40b-2251-4464-ab00-b276c7559302 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:11.297564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1176/2382] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 12:59:11 2022] PUT /identity/v3/projects/a4980c26d9b44dadbaa680d7ac6994d3/users/af4c6cd209c149ebb2d32fff17972715/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:59:14.499214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4b24e-b788-4e1c-bd16-420c4a9450d2 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:14.501086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4b24e-b788-4e1c-bd16-420c4a9450d2 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:14.501086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4b24e-b788-4e1c-bd16-420c4a9450d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:14.529923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-74a4b24e-b788-4e1c-bd16-420c4a9450d2 None None] MFA Rules not processed for user `4c96322c48c146c7b8bcb9959f533820`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:14.539192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-74a4b24e-b788-4e1c-bd16-420c4a9450d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:14.661118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1207/2383] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:59:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:59:20.215801 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cd17b-e6b1-498a-a87d-530d3d3cf9e0 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:20.216295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cd17b-e6b1-498a-a87d-530d3d3cf9e0 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:20.216295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cd17b-e6b1-498a-a87d-530d3d3cf9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:20.260626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-cc2cd17b-e6b1-498a-a87d-530d3d3cf9e0 None None] MFA Rules not processed for user `af4c6cd209c149ebb2d32fff17972715`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:20.263707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-cc2cd17b-e6b1-498a-a87d-530d3d3cf9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:20.278447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:59:20.395325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1177/2384] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:59:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:59:20.409423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4b24e-b788-4e1c-bd16-420c4a9450d2 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:20.481813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4b24e-b788-4e1c-bd16-420c4a9450d2 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:20.485525 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706515df-58b9-44e5-bfbf-a3ccea5210aa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff8fcb7fa2740bbb801badf800d2d6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:20.486302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706515df-58b9-44e5-bfbf-a3ccea5210aa service glance] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:20.486434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706515df-58b9-44e5-bfbf-a3ccea5210aa service glance] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:20.486748 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706515df-58b9-44e5-bfbf-a3ccea5210aa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:20.491062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:59:20.548238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706515df-58b9-44e5-bfbf-a3ccea5210aa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:20.549647 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706515df-58b9-44e5-bfbf-a3ccea5210aa service glance] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:20.559913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1208/2385] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:59:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:59:20.848100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076f48f-8c56-4e2a-980d-9d9a90efbcdd None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:20.848389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076f48f-8c56-4e2a-980d-9d9a90efbcdd None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:20.848664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076f48f-8c56-4e2a-980d-9d9a90efbcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:20.867811 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-8076f48f-8c56-4e2a-980d-9d9a90efbcdd None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:20.868981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-8076f48f-8c56-4e2a-980d-9d9a90efbcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:20.923707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1178/2386] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 12:59:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:59:20.936308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706515df-58b9-44e5-bfbf-a3ccea5210aa service glance] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:21.001938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706515df-58b9-44e5-bfbf-a3ccea5210aa service glance] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:21.006923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224554b5-7fbb-429a-ad1e-35d4582db88a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:21.007527 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224554b5-7fbb-429a-ad1e-35d4582db88a service swift] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:21.007808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224554b5-7fbb-429a-ad1e-35d4582db88a service swift] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:21.008084 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224554b5-7fbb-429a-ad1e-35d4582db88a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:21.016709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224554b5-7fbb-429a-ad1e-35d4582db88a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:21.018275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224554b5-7fbb-429a-ad1e-35d4582db88a service swift] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:21.026151 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1209/2387] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 12:59:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:59:30.916975 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=50 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:59:31.022971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=57 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:59:33.804314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959194f9-4c23-4dc1-ba2e-d0f57af8096e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:33.804906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959194f9-4c23-4dc1-ba2e-d0f57af8096e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:33.804906 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959194f9-4c23-4dc1-ba2e-d0f57af8096e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:33.827015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-959194f9-4c23-4dc1-ba2e-d0f57af8096e None None] MFA Rules not processed for user `eb144bc11f4b4cd59e1f70ebd6b7207c`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:33.828207 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-959194f9-4c23-4dc1-ba2e-d0f57af8096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:33.835816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1179/2388] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:59:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:59:36.970472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224554b5-7fbb-429a-ad1e-35d4582db88a service swift] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:36.975290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-224554b5-7fbb-429a-ad1e-35d4582db88a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:37.068051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224554b5-7fbb-429a-ad1e-35d4582db88a service swift] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:37.074371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67204cc-23d8-49c8-95c6-0a7281f318ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:37.075861 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67204cc-23d8-49c8-95c6-0a7281f318ea service placement] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:37.076311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67204cc-23d8-49c8-95c6-0a7281f318ea service placement] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:37.077426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67204cc-23d8-49c8-95c6-0a7281f318ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:37.079937 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d67204cc-23d8-49c8-95c6-0a7281f318ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:37.096087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67204cc-23d8-49c8-95c6-0a7281f318ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:37.098013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67204cc-23d8-49c8-95c6-0a7281f318ea service placement] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:37.159964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1210/2389] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:59:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:59:39.252564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef185b0-d51e-4b97-b5dc-890a104d6df6 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:39.253040 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef185b0-d51e-4b97-b5dc-890a104d6df6 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:39.253134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef185b0-d51e-4b97-b5dc-890a104d6df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:39.272860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-7ef185b0-d51e-4b97-b5dc-890a104d6df6 None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:39.274041 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7ef185b0-d51e-4b97-b5dc-890a104d6df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:39.282240 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1180/2390] 10.222.0.46 () {58 vars in 967 bytes} [Fri Sep 9 12:59:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:59:39.310636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67204cc-23d8-49c8-95c6-0a7281f318ea service placement] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:39.322363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67204cc-23d8-49c8-95c6-0a7281f318ea service placement] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:39.327634 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88885569-28d5-4ea4-b82a-2c6dc739af89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:39.328231 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88885569-28d5-4ea4-b82a-2c6dc739af89 service swift] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:39.328503 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88885569-28d5-4ea4-b82a-2c6dc739af89 service swift] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:39.328770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88885569-28d5-4ea4-b82a-2c6dc739af89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:39.336887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88885569-28d5-4ea4-b82a-2c6dc739af89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:39.338961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88885569-28d5-4ea4-b82a-2c6dc739af89 service swift] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:39.347073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1211/2391] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 12:59:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:59:44.952633 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e52ab-a581-4e88-8e7c-c44fbf9c21ed None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:44.953220 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e52ab-a581-4e88-8e7c-c44fbf9c21ed None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:44.953734 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e52ab-a581-4e88-8e7c-c44fbf9c21ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:44.984330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-711e52ab-a581-4e88-8e7c-c44fbf9c21ed None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:44.986035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-711e52ab-a581-4e88-8e7c-c44fbf9c21ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:44.994984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1181/2392] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 12:59:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:59:45.010815 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88885569-28d5-4ea4-b82a-2c6dc739af89 service swift] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:45.023864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88885569-28d5-4ea4-b82a-2c6dc739af89 service swift] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:45.029919 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6993f-b165-448a-8e9a-50f91ff0931b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:45.034614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6993f-b165-448a-8e9a-50f91ff0931b service swift] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:45.034614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6993f-b165-448a-8e9a-50f91ff0931b service swift] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:45.034614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6993f-b165-448a-8e9a-50f91ff0931b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:45.040154 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6993f-b165-448a-8e9a-50f91ff0931b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:45.042232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6993f-b165-448a-8e9a-50f91ff0931b service swift] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:45.053086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1212/2393] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 12:59:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:59:45.189653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e52ab-a581-4e88-8e7c-c44fbf9c21ed None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:59:45.275189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e52ab-a581-4e88-8e7c-c44fbf9c21ed None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:59:45.282306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d887c96-31dc-4cbe-bf0f-10d75e913ede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:59:45.282306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d887c96-31dc-4cbe-bf0f-10d75e913ede service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:45.282819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d887c96-31dc-4cbe-bf0f-10d75e913ede service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:45.283512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d887c96-31dc-4cbe-bf0f-10d75e913ede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:45.285761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7d887c96-31dc-4cbe-bf0f-10d75e913ede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:45.407834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d887c96-31dc-4cbe-bf0f-10d75e913ede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:59:45.412057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d887c96-31dc-4cbe-bf0f-10d75e913ede service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:59:45.428275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1182/2394] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 12:59:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:59:47.649684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1015fb9-0827-45f6-a273-fdcf5f18915c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:59:47.651211 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1015fb9-0827-45f6-a273-fdcf5f18915c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:59:47.651554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1015fb9-0827-45f6-a273-fdcf5f18915c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:59:47.675692 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b1015fb9-0827-45f6-a273-fdcf5f18915c None None] MFA Rules not processed for user `48f14caeb8f944c6aaf27913972869c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:59:47.677111 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b1015fb9-0827-45f6-a273-fdcf5f18915c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:59:47.754337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1213/2395] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 12:59:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:59:55.422216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=59 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:59:57.741578 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=69 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:06.570438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97548c54-0963-45fc-9166-a388ce972e53 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:06.570950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97548c54-0963-45fc-9166-a388ce972e53 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:06.571045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97548c54-0963-45fc-9166-a388ce972e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:06.593159 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-97548c54-0963-45fc-9166-a388ce972e53 None None] MFA Rules not processed for user `63a2902f465f4fd1aed80060c2669259`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:06.595351 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-97548c54-0963-45fc-9166-a388ce972e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:06.655926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1183/2396] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:00:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:00:16.646918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=25 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:16.725663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1015fb9-0827-45f6-a273-fdcf5f18915c None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:00:16.735830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1015fb9-0827-45f6-a273-fdcf5f18915c None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:00:16.739359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc0f6d-8150-44d5-845b-1b779d7e60f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:00:16.740453 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc0f6d-8150-44d5-845b-1b779d7e60f0 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:16.740453 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc0f6d-8150-44d5-845b-1b779d7e60f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:16.740453 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc0f6d-8150-44d5-845b-1b779d7e60f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:16.749455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc0f6d-8150-44d5-845b-1b779d7e60f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:00:16.750904 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc0f6d-8150-44d5-845b-1b779d7e60f0 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:00:16.760091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1214/2397] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:00:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:00:26.755988 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=6 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:00:33.694357 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e64060-abe7-4a73-b76a-c7eeb52c76a3 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:00:33.694920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e64060-abe7-4a73-b76a-c7eeb52c76a3 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:00:33.694920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e64060-abe7-4a73-b76a-c7eeb52c76a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:00:33.719597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-00e64060-abe7-4a73-b76a-c7eeb52c76a3 None None] MFA Rules not processed for user `6ec951604cde4768beb02906caf6d7c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:00:33.720914 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-00e64060-abe7-4a73-b76a-c7eeb52c76a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:00:33.781427 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1184/2398] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:00:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:00:43.774361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=25 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:25.135296 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc0f6d-8150-44d5-845b-1b779d7e60f0 service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:25.139479 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc0f6d-8150-44d5-845b-1b779d7e60f0 service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:25.143320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bedb6-94a3-40d8-b649-d467ad9d1287 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:25.144023 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bedb6-94a3-40d8-b649-d467ad9d1287 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:25.144363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bedb6-94a3-40d8-b649-d467ad9d1287 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:25.144848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bedb6-94a3-40d8-b649-d467ad9d1287 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:25.147659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-bd2bedb6-94a3-40d8-b649-d467ad9d1287 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:25.227112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bedb6-94a3-40d8-b649-d467ad9d1287 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:25.231938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bedb6-94a3-40d8-b649-d467ad9d1287 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:25.243244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1215/2399] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:01:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:01:26.045080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e64060-abe7-4a73-b76a-c7eeb52c76a3 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:26.055090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e64060-abe7-4a73-b76a-c7eeb52c76a3 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:26.058638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cba712-d5c9-4876-ba86-0140c719b4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:26.059480 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cba712-d5c9-4876-ba86-0140c719b4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:26.059957 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cba712-d5c9-4876-ba86-0140c719b4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:26.060441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cba712-d5c9-4876-ba86-0140c719b4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:26.063334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-85cba712-d5c9-4876-ba86-0140c719b4c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:26.079389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cba712-d5c9-4876-ba86-0140c719b4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:26.081072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cba712-d5c9-4876-ba86-0140c719b4c8 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:26.092265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1185/2400] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:01:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:01:26.229540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5895c-5dff-478c-b222-c0efdc235374 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:26.230100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5895c-5dff-478c-b222-c0efdc235374 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:26.230100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5895c-5dff-478c-b222-c0efdc235374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:26.249597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b5f5895c-5dff-478c-b222-c0efdc235374 None None] MFA Rules not processed for user `48f14caeb8f944c6aaf27913972869c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:26.250781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b5f5895c-5dff-478c-b222-c0efdc235374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:26.258223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1216/2401] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:01:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:01:30.584426 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36708045-2693-420a-940e-4b98534cc8ab None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:30.586406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36708045-2693-420a-940e-4b98534cc8ab None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:30.586406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36708045-2693-420a-940e-4b98534cc8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:30.604062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-36708045-2693-420a-940e-4b98534cc8ab None None] MFA Rules not processed for user `63a2902f465f4fd1aed80060c2669259`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:30.605242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-36708045-2693-420a-940e-4b98534cc8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:30.612544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1186/2402] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:01:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:01:36.243762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=37 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:38.175905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5895c-5dff-478c-b222-c0efdc235374 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:38.178189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b5f5895c-5dff-478c-b222-c0efdc235374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:38.195132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5895c-5dff-478c-b222-c0efdc235374 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:38.200947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ba296-d540-4448-b609-ab9d9589ad0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:38.201556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ba296-d540-4448-b609-ab9d9589ad0a service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:38.201835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ba296-d540-4448-b609-ab9d9589ad0a service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:38.202105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ba296-d540-4448-b609-ab9d9589ad0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:38.203864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d69ba296-d540-4448-b609-ab9d9589ad0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:38.215227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ba296-d540-4448-b609-ab9d9589ad0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:38.217297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ba296-d540-4448-b609-ab9d9589ad0a service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:38.226447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1217/2403] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:01:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:01:39.828909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36708045-2693-420a-940e-4b98534cc8ab None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:39.830680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-36708045-2693-420a-940e-4b98534cc8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:39.846587 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36708045-2693-420a-940e-4b98534cc8ab None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:39.852670 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba84afb-13ad-4aa1-a710-be43e7c03e01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:39.853262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba84afb-13ad-4aa1-a710-be43e7c03e01 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:39.853539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba84afb-13ad-4aa1-a710-be43e7c03e01 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:39.853807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba84afb-13ad-4aa1-a710-be43e7c03e01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:39.855498 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-dba84afb-13ad-4aa1-a710-be43e7c03e01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:39.866012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba84afb-13ad-4aa1-a710-be43e7c03e01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:39.867610 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba84afb-13ad-4aa1-a710-be43e7c03e01 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:39.876783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1187/2404] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:01:39 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:01:41.872042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690afc1-4288-47ea-8b3d-001ac599bd24 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:41.873051 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690afc1-4288-47ea-8b3d-001ac599bd24 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:41.873327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690afc1-4288-47ea-8b3d-001ac599bd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:41.893104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0690afc1-4288-47ea-8b3d-001ac599bd24 None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:01:41.894310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0690afc1-4288-47ea-8b3d-001ac599bd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:41.901998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1218/2405] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 13:01:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:01:41.942816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba84afb-13ad-4aa1-a710-be43e7c03e01 service nova] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:41.954248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba84afb-13ad-4aa1-a710-be43e7c03e01 service nova] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:41.959904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558236e9-90b0-448f-be8e-f1f7105cf4ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:41.960503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558236e9-90b0-448f-be8e-f1f7105cf4ee service swift] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:41.960785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558236e9-90b0-448f-be8e-f1f7105cf4ee service swift] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:41.961064 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558236e9-90b0-448f-be8e-f1f7105cf4ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:41.969202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558236e9-90b0-448f-be8e-f1f7105cf4ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:41.970812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558236e9-90b0-448f-be8e-f1f7105cf4ee service swift] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:41.979523 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1188/2406] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 13:01:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:01:51.887983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=9 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:51.976236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=21 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:01:57.034723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690afc1-4288-47ea-8b3d-001ac599bd24 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:57.046971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690afc1-4288-47ea-8b3d-001ac599bd24 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:57.050717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9e131-2b20-425e-853e-a1117615262e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:57.051329 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e131-2b20-425e-853e-a1117615262e service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:57.051602 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e131-2b20-425e-853e-a1117615262e service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:57.052293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e131-2b20-425e-853e-a1117615262e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:57.053690 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6bd9e131-2b20-425e-853e-a1117615262e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:57.064733 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9e131-2b20-425e-853e-a1117615262e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:57.066189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9e131-2b20-425e-853e-a1117615262e service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:57.075209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1219/2407] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:01:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:01:58.425814 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558236e9-90b0-448f-be8e-f1f7105cf4ee service swift] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:01:58.438657 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558236e9-90b0-448f-be8e-f1f7105cf4ee service swift] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:01:58.444733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731446ea-3507-4279-bff5-f67bc9f0c232 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:01:58.445463 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731446ea-3507-4279-bff5-f67bc9f0c232 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:01:58.445821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731446ea-3507-4279-bff5-f67bc9f0c232 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:01:58.446141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731446ea-3507-4279-bff5-f67bc9f0c232 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:01:58.448319 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-731446ea-3507-4279-bff5-f67bc9f0c232 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:01:58.461153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731446ea-3507-4279-bff5-f67bc9f0c232 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:01:58.463371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731446ea-3507-4279-bff5-f67bc9f0c232 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:01:58.475094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1189/2408] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:01:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:02:02.160910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f980c1-9a69-40a8-b353-e58312e991e7 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:02.161372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f980c1-9a69-40a8-b353-e58312e991e7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:02.161723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f980c1-9a69-40a8-b353-e58312e991e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:02.188871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-87f980c1-9a69-40a8-b353-e58312e991e7 None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:02.190160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-87f980c1-9a69-40a8-b353-e58312e991e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:02.197887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1220/2409] 10.222.0.46 () {58 vars in 967 bytes} [Fri Sep 9 13:02:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:02.211487 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731446ea-3507-4279-bff5-f67bc9f0c232 service nova] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:02.225763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731446ea-3507-4279-bff5-f67bc9f0c232 service nova] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:02.231778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd0581-8da3-44cf-8140-1ba2bc2a8dd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:02.235294 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd0581-8da3-44cf-8140-1ba2bc2a8dd5 service swift] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:02.235294 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd0581-8da3-44cf-8140-1ba2bc2a8dd5 service swift] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:02.235294 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd0581-8da3-44cf-8140-1ba2bc2a8dd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:02.241392 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd0581-8da3-44cf-8140-1ba2bc2a8dd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:02.243001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd0581-8da3-44cf-8140-1ba2bc2a8dd5 service swift] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:02.252115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1190/2410] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 13:02:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 13:02:02.700303 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31717ad-e182-4d0e-8522-5c706f295487 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:02.700591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31717ad-e182-4d0e-8522-5c706f295487 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:02.700936 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31717ad-e182-4d0e-8522-5c706f295487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:02.719788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b31717ad-e182-4d0e-8522-5c706f295487 None None] MFA Rules not processed for user `4c96322c48c146c7b8bcb9959f533820`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:02.720965 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b31717ad-e182-4d0e-8522-5c706f295487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:02.729717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1221/2411] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:03.083047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd0581-8da3-44cf-8140-1ba2bc2a8dd5 service swift] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:03.093397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd0581-8da3-44cf-8140-1ba2bc2a8dd5 service swift] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:03.097183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b892e-9cd9-4c51-82a7-0f4f7b26056c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:03.097798 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b892e-9cd9-4c51-82a7-0f4f7b26056c service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.098080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b892e-9cd9-4c51-82a7-0f4f7b26056c service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.098355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b892e-9cd9-4c51-82a7-0f4f7b26056c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.100229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b87b892e-9cd9-4c51-82a7-0f4f7b26056c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:03.111467 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b892e-9cd9-4c51-82a7-0f4f7b26056c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:03.112894 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b892e-9cd9-4c51-82a7-0f4f7b26056c service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:03.125457 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1191/2412] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:02:03.286248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2264b5-35ec-473b-83f2-4eae1093d345 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.286819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2264b5-35ec-473b-83f2-4eae1093d345 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.286950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2264b5-35ec-473b-83f2-4eae1093d345 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.289771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1222/2413] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 13:02:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:03.305234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689b952-e6ba-4db7-916d-8b14f5f3f5e5 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.305813 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689b952-e6ba-4db7-916d-8b14f5f3f5e5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.305813 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689b952-e6ba-4db7-916d-8b14f5f3f5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.342816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-b689b952-e6ba-4db7-916d-8b14f5f3f5e5 None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:03.342816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b689b952-e6ba-4db7-916d-8b14f5f3f5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:03.352970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1192/2414] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 13:02:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:03.365077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2264b5-35ec-473b-83f2-4eae1093d345 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:03.377870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2264b5-35ec-473b-83f2-4eae1093d345 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:03.381971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d789daa-1fd1-4497-a92a-4ca4d9e96e01 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eff8fcb7fa2740bbb801badf800d2d6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:03.382622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d789daa-1fd1-4497-a92a-4ca4d9e96e01 service glance] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.383109 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d789daa-1fd1-4497-a92a-4ca4d9e96e01 service glance] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.383384 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d789daa-1fd1-4497-a92a-4ca4d9e96e01 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.394776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d789daa-1fd1-4497-a92a-4ca4d9e96e01 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:03.396386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d789daa-1fd1-4497-a92a-4ca4d9e96e01 service glance] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:03.408285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1223/2415] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:02:03.624733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5f314-ef2e-4c36-98ce-174ba9bb13fd None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.625211 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5f314-ef2e-4c36-98ce-174ba9bb13fd None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.625491 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5f314-ef2e-4c36-98ce-174ba9bb13fd None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.626953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1193/2416] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 13:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:03.632052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bd9c4-2e41-4a24-ba19-015752fd595e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.632418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bd9c4-2e41-4a24-ba19-015752fd595e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.632688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bd9c4-2e41-4a24-ba19-015752fd595e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.659451 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-df6bd9c4-2e41-4a24-ba19-015752fd595e None None] MFA Rules not processed for user `e46bd9d49bf64013ae58770b0284d81e`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:03.661286 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-df6bd9c4-2e41-4a24-ba19-015752fd595e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:03.672330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1224/2417] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 13:02:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:02:03.685647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5f314-ef2e-4c36-98ce-174ba9bb13fd None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:03.698031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5f314-ef2e-4c36-98ce-174ba9bb13fd None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:03.701628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781efaa1-501f-4d9b-8d33-2e0fdee4e839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:03.702217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781efaa1-501f-4d9b-8d33-2e0fdee4e839 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.702508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781efaa1-501f-4d9b-8d33-2e0fdee4e839 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.702828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781efaa1-501f-4d9b-8d33-2e0fdee4e839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.712471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781efaa1-501f-4d9b-8d33-2e0fdee4e839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:03.712471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781efaa1-501f-4d9b-8d33-2e0fdee4e839 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:03.721606 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1194/2418] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:02:03.815986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ef721-8390-4edf-9a9f-cec600ba6954 None None] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.816304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ef721-8390-4edf-9a9f-cec600ba6954 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.816531 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ef721-8390-4edf-9a9f-cec600ba6954 None None] PATH_INFO: `/` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.818032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1225/2419] 10.222.0.46 () {54 vars in 921 bytes} [Fri Sep 9 13:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:03.832417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebc10c-f061-44a0-8d98-9900bdd93c8f None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.834869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebc10c-f061-44a0-8d98-9900bdd93c8f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.841658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebc10c-f061-44a0-8d98-9900bdd93c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.866139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-c0ebc10c-f061-44a0-8d98-9900bdd93c8f None None] MFA Rules not processed for user `f29af98c5fba4569b16edf240b13c5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:03.867450 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c0ebc10c-f061-44a0-8d98-9900bdd93c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:03.878298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ef721-8390-4edf-9a9f-cec600ba6954 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:03.880278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1195/2420] 10.222.0.46 () {58 vars in 1018 bytes} [Fri Sep 9 13:02:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:03.891691 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebc10c-f061-44a0-8d98-9900bdd93c8f None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:03.904232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ef721-8390-4edf-9a9f-cec600ba6954 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:03.904937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebc10c-f061-44a0-8d98-9900bdd93c8f None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:03.908821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e571a-0fda-48d9-b47b-1f0e743243a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:03.908972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d2d29-a82a-4538-bbe6-e07d3dee81d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:03.909432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e571a-0fda-48d9-b47b-1f0e743243a7 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.909590 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d2d29-a82a-4538-bbe6-e07d3dee81d6 service placement] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:03.909969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d2d29-a82a-4538-bbe6-e07d3dee81d6 service placement] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.910084 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e571a-0fda-48d9-b47b-1f0e743243a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:03.910240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d2d29-a82a-4538-bbe6-e07d3dee81d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.910380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e571a-0fda-48d9-b47b-1f0e743243a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:03.912255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-891d2d29-a82a-4538-bbe6-e07d3dee81d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:03.923178 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e571a-0fda-48d9-b47b-1f0e743243a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:03.924940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e571a-0fda-48d9-b47b-1f0e743243a7 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:03.925854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d2d29-a82a-4538-bbe6-e07d3dee81d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:03.927543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d2d29-a82a-4538-bbe6-e07d3dee81d6 service placement] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:03.938642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1226/2421] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:02:03.939232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1196/2422] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:02:07.457744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d72fec-7643-4781-acb6-f8e3fa8c5bb2 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:07.457744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d72fec-7643-4781-acb6-f8e3fa8c5bb2 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:07.457744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d72fec-7643-4781-acb6-f8e3fa8c5bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:07.483508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-17d72fec-7643-4781-acb6-f8e3fa8c5bb2 None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:07.485323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-17d72fec-7643-4781-acb6-f8e3fa8c5bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:07.512960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1197/2423] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 13:02:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:02:07.526699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d2d29-a82a-4538-bbe6-e07d3dee81d6 service placement] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:07.543818 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d2d29-a82a-4538-bbe6-e07d3dee81d6 service placement] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:07.563796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d5dba-0f45-4bfc-9da6-2642f0e85120 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:07.564898 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d5dba-0f45-4bfc-9da6-2642f0e85120 service swift] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:07.566733 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d5dba-0f45-4bfc-9da6-2642f0e85120 service swift] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:07.568520 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d5dba-0f45-4bfc-9da6-2642f0e85120 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:07.582197 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d5dba-0f45-4bfc-9da6-2642f0e85120 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:07.584208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d5dba-0f45-4bfc-9da6-2642f0e85120 service swift] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:07.603494 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1227/2424] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 13:02:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:02:13.199278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d72fec-7643-4781-acb6-f8e3fa8c5bb2 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:13.217422 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d72fec-7643-4781-acb6-f8e3fa8c5bb2 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:13.240726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293dab9-a1e7-4ddb-9e91-b0d6a53b009f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:13.241578 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293dab9-a1e7-4ddb-9e91-b0d6a53b009f service placement] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:13.241889 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293dab9-a1e7-4ddb-9e91-b0d6a53b009f service placement] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:13.242363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293dab9-a1e7-4ddb-9e91-b0d6a53b009f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:13.248324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-1293dab9-a1e7-4ddb-9e91-b0d6a53b009f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:13.279175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293dab9-a1e7-4ddb-9e91-b0d6a53b009f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:13.279969 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293dab9-a1e7-4ddb-9e91-b0d6a53b009f service placement] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:13.294764 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1198/2425] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:02:13.367897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d5dba-0f45-4bfc-9da6-2642f0e85120 service swift] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:13.449080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d5dba-0f45-4bfc-9da6-2642f0e85120 service swift] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:13.453554 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:13.454183 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:13.454463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:13.454772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] PATH_INFO: `/v3/users/eb144bc11f4b4cd59e1f70ebd6b7207c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:13.456148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb144bc11f4b4cd59e1f70ebd6b7207c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:13.458032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:13.601569 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb144bc11f4b4cd59e1f70ebd6b7207c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:13.668506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb144bc11f4b4cd59e1f70ebd6b7207c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:13.669235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb144bc11f4b4cd59e1f70ebd6b7207c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:13.669866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb144bc11f4b4cd59e1f70ebd6b7207c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:13.683404 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb144bc11f4b4cd59e1f70ebd6b7207c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:13.715470 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1228/2426] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:02:13 2022] DELETE /identity/v3/users/eb144bc11f4b4cd59e1f70ebd6b7207c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:14.042953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293dab9-a1e7-4ddb-9e91-b0d6a53b009f service placement] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:14.058149 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293dab9-a1e7-4ddb-9e91-b0d6a53b009f service placement] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:14.157290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:14.158116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:14.158405 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:14.158710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] PATH_INFO: `/v3/projects/bfb3240493c84bda8102619400759342` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:14.160045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfb3240493c84bda8102619400759342)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:14.162331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:14.283002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfb3240493c84bda8102619400759342', 'request_id': 'req-351ff4da-c78b-4b7f-a685-4d1b76eb209f'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:14.289201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfb3240493c84bda8102619400759342', 'request_id': 'req-351ff4da-c78b-4b7f-a685-4d1b76eb209f'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:14.422125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfb3240493c84bda8102619400759342', 'request_id': 'req-351ff4da-c78b-4b7f-a685-4d1b76eb209f'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:14.424719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:14.436176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] The token cache is being invalidate because project bfb3240493c84bda8102619400759342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:02:14.436554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:14.437213 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:14.441593 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1199/2427] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:02:14 2022] DELETE /identity/v3/projects/bfb3240493c84bda8102619400759342 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:14.442793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:14.444303 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:14.460571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da98466-0687-4bdd-b8b6-c83380d55f6b admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:14.465644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:14.524295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a7a03-c898-4dd7-8ff6-783d0bce08df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:14.524953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a7a03-c898-4dd7-8ff6-783d0bce08df admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:14.525237 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a7a03-c898-4dd7-8ff6-783d0bce08df admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:14.525566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a7a03-c898-4dd7-8ff6-783d0bce08df admin admin] PATH_INFO: `/v3/users/af4c6cd209c149ebb2d32fff17972715` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:14.526874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a7a03-c898-4dd7-8ff6-783d0bce08df admin admin] RBAC: Authorizing `identity:delete_user(user_id=af4c6cd209c149ebb2d32fff17972715)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:14.528878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a7a03-c898-4dd7-8ff6-783d0bce08df admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:14.740690 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-023a7a03-c898-4dd7-8ff6-783d0bce08df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af4c6cd209c149ebb2d32fff17972715'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:14.827774 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-023a7a03-c898-4dd7-8ff6-783d0bce08df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af4c6cd209c149ebb2d32fff17972715'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:14.835482 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-023a7a03-c898-4dd7-8ff6-783d0bce08df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af4c6cd209c149ebb2d32fff17972715'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:14.835482 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-023a7a03-c898-4dd7-8ff6-783d0bce08df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af4c6cd209c149ebb2d32fff17972715'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:14.847105 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-023a7a03-c898-4dd7-8ff6-783d0bce08df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af4c6cd209c149ebb2d32fff17972715'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:14.860819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1229/2428] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:02:14 2022] DELETE /identity/v3/users/af4c6cd209c149ebb2d32fff17972715 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:02:15.132714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:15.134185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:15.156339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ff4da-c78b-4b7f-a685-4d1b76eb209f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:15.231462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753e549-9961-4e35-8ac0-e170df0a002e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:15.232281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753e549-9961-4e35-8ac0-e170df0a002e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:15.232589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753e549-9961-4e35-8ac0-e170df0a002e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:15.232929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753e549-9961-4e35-8ac0-e170df0a002e admin admin] PATH_INFO: `/v3/projects/a4980c26d9b44dadbaa680d7ac6994d3` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:15.234143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753e549-9961-4e35-8ac0-e170df0a002e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4980c26d9b44dadbaa680d7ac6994d3)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:15.236161 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753e549-9961-4e35-8ac0-e170df0a002e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:15.371128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4753e549-9961-4e35-8ac0-e170df0a002e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4980c26d9b44dadbaa680d7ac6994d3', 'request_id': 'req-4753e549-9961-4e35-8ac0-e170df0a002e'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:15.381556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4753e549-9961-4e35-8ac0-e170df0a002e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4980c26d9b44dadbaa680d7ac6994d3', 'request_id': 'req-4753e549-9961-4e35-8ac0-e170df0a002e'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:15.549555 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4753e549-9961-4e35-8ac0-e170df0a002e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4980c26d9b44dadbaa680d7ac6994d3', 'request_id': 'req-4753e549-9961-4e35-8ac0-e170df0a002e'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:15.561888 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4753e549-9961-4e35-8ac0-e170df0a002e admin admin] The token cache is being invalidate because project a4980c26d9b44dadbaa680d7ac6994d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:02:15.562232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4753e549-9961-4e35-8ac0-e170df0a002e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:15.562955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4753e549-9961-4e35-8ac0-e170df0a002e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:15.564455 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1200/2429] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:02:15 2022] DELETE /identity/v3/projects/a4980c26d9b44dadbaa680d7ac6994d3 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:15.610452 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094434f1-9f78-42a4-a6d5-923240f8fcc5 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:15.611044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094434f1-9f78-42a4-a6d5-923240f8fcc5 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:15.611044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094434f1-9f78-42a4-a6d5-923240f8fcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:15.637844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-094434f1-9f78-42a4-a6d5-923240f8fcc5 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:15.639373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-094434f1-9f78-42a4-a6d5-923240f8fcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:15.704638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1230/2430] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:15.712567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651c1b-e51b-4632-9f1f-087f3824a217 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:15.712895 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651c1b-e51b-4632-9f1f-087f3824a217 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:15.713174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651c1b-e51b-4632-9f1f-087f3824a217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:15.739513 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-00651c1b-e51b-4632-9f1f-087f3824a217 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:15.740819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-00651c1b-e51b-4632-9f1f-087f3824a217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:15.750843 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1201/2431] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:15.755215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094434f1-9f78-42a4-a6d5-923240f8fcc5 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:15.765838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094434f1-9f78-42a4-a6d5-923240f8fcc5 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:15.770792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948cc00-448a-4a65-b4d4-ddf7b8333b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:15.771417 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948cc00-448a-4a65-b4d4-ddf7b8333b03 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:15.771702 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948cc00-448a-4a65-b4d4-ddf7b8333b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:15.771969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948cc00-448a-4a65-b4d4-ddf7b8333b03 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:15.772628 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948cc00-448a-4a65-b4d4-ddf7b8333b03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:15.772981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948cc00-448a-4a65-b4d4-ddf7b8333b03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:15.774585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948cc00-448a-4a65-b4d4-ddf7b8333b03 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:15.788917 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1231/2432] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:15.793909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00651c1b-e51b-4632-9f1f-087f3824a217 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:15.804266 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00651c1b-e51b-4632-9f1f-087f3824a217 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:15.811025 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82c7ca-f3d5-4a42-bda5-8150051cefd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:15.815788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82c7ca-f3d5-4a42-bda5-8150051cefd6 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:15.815788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82c7ca-f3d5-4a42-bda5-8150051cefd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:15.815788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82c7ca-f3d5-4a42-bda5-8150051cefd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:15.815788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82c7ca-f3d5-4a42-bda5-8150051cefd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:15.815788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82c7ca-f3d5-4a42-bda5-8150051cefd6 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:15.901946 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1202/2433] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:02:15 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:15.903171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948cc00-448a-4a65-b4d4-ddf7b8333b03 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:15.919943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948cc00-448a-4a65-b4d4-ddf7b8333b03 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:15.953546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:16.035552 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07a25f-cf1c-40cf-a981-4becb57a1f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.036421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07a25f-cf1c-40cf-a981-4becb57a1f6f admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.036785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07a25f-cf1c-40cf-a981-4becb57a1f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.037177 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07a25f-cf1c-40cf-a981-4becb57a1f6f admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.038188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07a25f-cf1c-40cf-a981-4becb57a1f6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:16.046494 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07a25f-cf1c-40cf-a981-4becb57a1f6f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:16.142517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1232/2434] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:15 2022] POST /identity/v3/users => generated 287 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:16.144052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82c7ca-f3d5-4a42-bda5-8150051cefd6 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:16.155533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82c7ca-f3d5-4a42-bda5-8150051cefd6 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:16.160451 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd9298-72c7-4c5c-b5d0-e0252e5252b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.161065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd9298-72c7-4c5c-b5d0-e0252e5252b0 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.161419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd9298-72c7-4c5c-b5d0-e0252e5252b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.162526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd9298-72c7-4c5c-b5d0-e0252e5252b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.162526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd9298-72c7-4c5c-b5d0-e0252e5252b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:16.164205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd9298-72c7-4c5c-b5d0-e0252e5252b0 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:16.176206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1203/2435] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:02:16.180193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07a25f-cf1c-40cf-a981-4becb57a1f6f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:16.191642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07a25f-cf1c-40cf-a981-4becb57a1f6f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:16.196481 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a5e47-2db6-4917-9458-fb3a78e3e658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.197877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a5e47-2db6-4917-9458-fb3a78e3e658 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.198310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a5e47-2db6-4917-9458-fb3a78e3e658 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.198697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a5e47-2db6-4917-9458-fb3a78e3e658 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.199422 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a5e47-2db6-4917-9458-fb3a78e3e658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:16.201094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a5e47-2db6-4917-9458-fb3a78e3e658 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:16.213711 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1233/2436] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:02:16.218054 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd9298-72c7-4c5c-b5d0-e0252e5252b0 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:16.229480 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd9298-72c7-4c5c-b5d0-e0252e5252b0 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:16.235218 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f175c-ec82-4e6f-9a83-578dd1708508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.235883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f175c-ec82-4e6f-9a83-578dd1708508 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.236232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f175c-ec82-4e6f-9a83-578dd1708508 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.236640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f175c-ec82-4e6f-9a83-578dd1708508 admin admin] PATH_INFO: `/v3/projects/add9aa988a0848d185d4f06039cfd00e/users/a112ff54558147cf8739322b085fe1ab/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.252465 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f175c-ec82-4e6f-9a83-578dd1708508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=add9aa988a0848d185d4f06039cfd00e, user_id=a112ff54558147cf8739322b085fe1ab, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:16.255231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f175c-ec82-4e6f-9a83-578dd1708508 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:16.355209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1204/2437] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:02:16 2022] PUT /identity/v3/projects/add9aa988a0848d185d4f06039cfd00e/users/a112ff54558147cf8739322b085fe1ab/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:16.360517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c148ea-f2f7-4fa7-aff5-94678ddbae28 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.360798 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c148ea-f2f7-4fa7-aff5-94678ddbae28 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.361070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c148ea-f2f7-4fa7-aff5-94678ddbae28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.383023 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-96c148ea-f2f7-4fa7-aff5-94678ddbae28 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:16.384315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-96c148ea-f2f7-4fa7-aff5-94678ddbae28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:16.473776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1234/2438] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:16.474865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f175c-ec82-4e6f-9a83-578dd1708508 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:16.474865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f61f175c-ec82-4e6f-9a83-578dd1708508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:16.491819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:16.530403 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f175c-ec82-4e6f-9a83-578dd1708508 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:16.537464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d918c-d1da-4aca-bff5-fdac6563a973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:16.537464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d918c-d1da-4aca-bff5-fdac6563a973 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:16.537464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d918c-d1da-4aca-bff5-fdac6563a973 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:16.537464 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d918c-d1da-4aca-bff5-fdac6563a973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:16.546212 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d918c-d1da-4aca-bff5-fdac6563a973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:16.547766 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d918c-d1da-4aca-bff5-fdac6563a973 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:16.561456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1205/2439] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:02:22.719258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c148ea-f2f7-4fa7-aff5-94678ddbae28 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:22.721103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-96c148ea-f2f7-4fa7-aff5-94678ddbae28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:22.826354 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c148ea-f2f7-4fa7-aff5-94678ddbae28 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:22.833781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6328b8-f5ce-496c-9fe7-3931dbe6417f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:22.834398 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6328b8-f5ce-496c-9fe7-3931dbe6417f service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:22.834725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6328b8-f5ce-496c-9fe7-3931dbe6417f service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:22.835002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6328b8-f5ce-496c-9fe7-3931dbe6417f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:22.840644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9b6328b8-f5ce-496c-9fe7-3931dbe6417f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:22.863355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6328b8-f5ce-496c-9fe7-3931dbe6417f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:22.864927 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6328b8-f5ce-496c-9fe7-3931dbe6417f service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:22.895921 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1235/2440] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:02:23.324209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687c9bc-a454-4187-b71a-b04f6bf149eb None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:23.330424 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687c9bc-a454-4187-b71a-b04f6bf149eb None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:23.330424 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687c9bc-a454-4187-b71a-b04f6bf149eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:23.397201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-f687c9bc-a454-4187-b71a-b04f6bf149eb None None] MFA Rules not processed for user `a112ff54558147cf8739322b085fe1ab`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:23.399401 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f687c9bc-a454-4187-b71a-b04f6bf149eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:23.530390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1206/2441] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:23.534468 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6328b8-f5ce-496c-9fe7-3931dbe6417f service nova] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:23.545958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6328b8-f5ce-496c-9fe7-3931dbe6417f service nova] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:23.550603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cbc6f-db99-4a36-8432-e8193e8d4dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:23.551202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cbc6f-db99-4a36-8432-e8193e8d4dec admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:23.551483 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cbc6f-db99-4a36-8432-e8193e8d4dec admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:23.551752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cbc6f-db99-4a36-8432-e8193e8d4dec admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:23.552623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cbc6f-db99-4a36-8432-e8193e8d4dec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:23.554588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cbc6f-db99-4a36-8432-e8193e8d4dec admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:23.574158 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:23.681034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1236/2442] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:02:23 2022] POST /identity/v3/projects => generated 371 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:23.689218 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687c9bc-a454-4187-b71a-b04f6bf149eb None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:23.698696 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687c9bc-a454-4187-b71a-b04f6bf149eb None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:23.782661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e5373-87b6-4e67-975f-a06d06a30419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:23.784139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e5373-87b6-4e67-975f-a06d06a30419 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:23.784139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e5373-87b6-4e67-975f-a06d06a30419 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:23.784139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e5373-87b6-4e67-975f-a06d06a30419 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:23.785010 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e5373-87b6-4e67-975f-a06d06a30419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:23.786831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e5373-87b6-4e67-975f-a06d06a30419 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:23.866592 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1207/2443] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:23 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:23.867510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cbc6f-db99-4a36-8432-e8193e8d4dec admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:23.879219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cbc6f-db99-4a36-8432-e8193e8d4dec admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:23.883966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfe6f5-45a3-42d4-80dd-231b9d6b5f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:23.885353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfe6f5-45a3-42d4-80dd-231b9d6b5f0b admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:23.885632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfe6f5-45a3-42d4-80dd-231b9d6b5f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:23.885903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfe6f5-45a3-42d4-80dd-231b9d6b5f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:23.886606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfe6f5-45a3-42d4-80dd-231b9d6b5f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:23.888184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfe6f5-45a3-42d4-80dd-231b9d6b5f0b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:23.898924 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1237/2444] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:02:23.904073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e5373-87b6-4e67-975f-a06d06a30419 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:23.914512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e5373-87b6-4e67-975f-a06d06a30419 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:23.919701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618cbaa-e100-4e8c-9729-66013ef8609d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:23.920351 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618cbaa-e100-4e8c-9729-66013ef8609d admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:23.920628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618cbaa-e100-4e8c-9729-66013ef8609d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:23.920924 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618cbaa-e100-4e8c-9729-66013ef8609d admin admin] PATH_INFO: `/v3/projects/955441331aa64ee7b01ffbbb7016915c/users/e8c4e93409dd40ad85db4f6c5bfe8e9a/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:23.938275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618cbaa-e100-4e8c-9729-66013ef8609d admin admin] RBAC: Authorizing `identity:create_grant(project_id=955441331aa64ee7b01ffbbb7016915c, user_id=e8c4e93409dd40ad85db4f6c5bfe8e9a, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:23.940994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618cbaa-e100-4e8c-9729-66013ef8609d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:24.030874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1208/2445] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:02:23 2022] PUT /identity/v3/projects/955441331aa64ee7b01ffbbb7016915c/users/e8c4e93409dd40ad85db4f6c5bfe8e9a/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:24.035256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfe6f5-45a3-42d4-80dd-231b9d6b5f0b admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:24.048824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfe6f5-45a3-42d4-80dd-231b9d6b5f0b admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:24.132146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78e09c-779c-456d-b32c-bfed944a2404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:24.132854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78e09c-779c-456d-b32c-bfed944a2404 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:24.133202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78e09c-779c-456d-b32c-bfed944a2404 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:24.133543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78e09c-779c-456d-b32c-bfed944a2404 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:24.134256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78e09c-779c-456d-b32c-bfed944a2404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:24.136589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78e09c-779c-456d-b32c-bfed944a2404 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:24.150922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1238/2446] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:02:24.153216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618cbaa-e100-4e8c-9729-66013ef8609d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:24.168257 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618cbaa-e100-4e8c-9729-66013ef8609d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:24.176216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc703b26-7633-4529-9254-6bf1e8832d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:24.177018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc703b26-7633-4529-9254-6bf1e8832d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:24.177303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc703b26-7633-4529-9254-6bf1e8832d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:24.177578 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc703b26-7633-4529-9254-6bf1e8832d10 admin admin] PATH_INFO: `/v3/projects/955441331aa64ee7b01ffbbb7016915c/users/e8c4e93409dd40ad85db4f6c5bfe8e9a/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:24.180340 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc703b26-7633-4529-9254-6bf1e8832d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=955441331aa64ee7b01ffbbb7016915c, user_id=e8c4e93409dd40ad85db4f6c5bfe8e9a, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:24.183307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc703b26-7633-4529-9254-6bf1e8832d10 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:24.194634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1209/2447] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:02:24 2022] PUT /identity/v3/projects/955441331aa64ee7b01ffbbb7016915c/users/e8c4e93409dd40ad85db4f6c5bfe8e9a/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:25.241582 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00833978-0874-4fe8-9bda-ac7b42b0d8fb None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.243096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00833978-0874-4fe8-9bda-ac7b42b0d8fb None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.244570 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00833978-0874-4fe8-9bda-ac7b42b0d8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.293741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-00833978-0874-4fe8-9bda-ac7b42b0d8fb None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:25.296267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-00833978-0874-4fe8-9bda-ac7b42b0d8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:25.432045 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1239/2448] 10.222.0.46 () {58 vars in 967 bytes} [Fri Sep 9 13:02:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:25.450982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc703b26-7633-4529-9254-6bf1e8832d10 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:25.452919 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-fc703b26-7633-4529-9254-6bf1e8832d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:25.458868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:25.581380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc703b26-7633-4529-9254-6bf1e8832d10 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:25.588197 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc16a7d-cc03-4298-b086-c2a0315ff3cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:25.588992 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc16a7d-cc03-4298-b086-c2a0315ff3cc service swift] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:25.589994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc16a7d-cc03-4298-b086-c2a0315ff3cc service swift] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:25.589994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc16a7d-cc03-4298-b086-c2a0315ff3cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:25.601958 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc16a7d-cc03-4298-b086-c2a0315ff3cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:25.605352 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc16a7d-cc03-4298-b086-c2a0315ff3cc service swift] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:25.615615 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1210/2449] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 13:02:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:02:31.552379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03960d4-8fc6-484f-b7db-f88b4accfd72 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:31.552909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03960d4-8fc6-484f-b7db-f88b4accfd72 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:31.553118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03960d4-8fc6-484f-b7db-f88b4accfd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:31.586379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-a03960d4-8fc6-484f-b7db-f88b4accfd72 None None] MFA Rules not processed for user `e8c4e93409dd40ad85db4f6c5bfe8e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:31.587802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a03960d4-8fc6-484f-b7db-f88b4accfd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:31.665463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1240/2450] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:31.679965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc16a7d-cc03-4298-b086-c2a0315ff3cc service swift] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:31.760955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc16a7d-cc03-4298-b086-c2a0315ff3cc service swift] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:31.766948 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ebd96-3887-4097-95bc-45b6b3f26a43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:31.767569 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ebd96-3887-4097-95bc-45b6b3f26a43 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:31.770568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ebd96-3887-4097-95bc-45b6b3f26a43 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:31.770568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ebd96-3887-4097-95bc-45b6b3f26a43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:31.830471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ebd96-3887-4097-95bc-45b6b3f26a43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:31.832383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ebd96-3887-4097-95bc-45b6b3f26a43 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:31.842960 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1211/2451] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:02:34.631351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c603c-70bb-4c14-bf88-71c254669f8e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:34.632006 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c603c-70bb-4c14-bf88-71c254669f8e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:34.632006 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c603c-70bb-4c14-bf88-71c254669f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:34.653446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-f46c603c-70bb-4c14-bf88-71c254669f8e None None] MFA Rules not processed for user `a112ff54558147cf8739322b085fe1ab`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:34.654941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f46c603c-70bb-4c14-bf88-71c254669f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:34.662076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1241/2452] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:40.098296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ebd96-3887-4097-95bc-45b6b3f26a43 service nova] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:40.113472 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ebd96-3887-4097-95bc-45b6b3f26a43 service nova] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:40.131060 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab45ff-adfc-487d-a159-d3b06e4ab833 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:40.131809 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab45ff-adfc-487d-a159-d3b06e4ab833 service nova] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:40.132153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab45ff-adfc-487d-a159-d3b06e4ab833 service nova] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:40.132495 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab45ff-adfc-487d-a159-d3b06e4ab833 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:40.135039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d0ab45ff-adfc-487d-a159-d3b06e4ab833 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:40.148647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab45ff-adfc-487d-a159-d3b06e4ab833 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:40.151094 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab45ff-adfc-487d-a159-d3b06e4ab833 service nova] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:40.163649 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1212/2453] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:02:44.647359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:46.611502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c603c-70bb-4c14-bf88-71c254669f8e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:46.679081 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c603c-70bb-4c14-bf88-71c254669f8e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:46.684302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf29b4-24e5-4496-a9ce-0ae23499d0df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:46.685126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf29b4-24e5-4496-a9ce-0ae23499d0df service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:46.685496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf29b4-24e5-4496-a9ce-0ae23499d0df service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:46.685862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf29b4-24e5-4496-a9ce-0ae23499d0df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:46.687809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ddcf29b4-24e5-4496-a9ce-0ae23499d0df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:46.767382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf29b4-24e5-4496-a9ce-0ae23499d0df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:46.768814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf29b4-24e5-4496-a9ce-0ae23499d0df service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:46.779153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1242/2454] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:02:50.158193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=93 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:50.430953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9519d3-7f62-413b-a1a3-9f98c6a23ab8 None None] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:50.431298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9519d3-7f62-413b-a1a3-9f98c6a23ab8 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:50.431519 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9519d3-7f62-413b-a1a3-9f98c6a23ab8 None None] PATH_INFO: `/` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:50.434061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1213/2455] 10.222.0.46 () {54 vars in 961 bytes} [Fri Sep 9 13:02:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 13:02:50.439168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddc91d-616f-4d36-9597-d728561d57e4 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:50.439668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddc91d-616f-4d36-9597-d728561d57e4 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:50.439760 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddc91d-616f-4d36-9597-d728561d57e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:50.462662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-80ddc91d-616f-4d36-9597-d728561d57e4 None None] MFA Rules not processed for user `33e177d86450489cacc86578ba26509d`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:50.474283 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-80ddc91d-616f-4d36-9597-d728561d57e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:50.552246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9519d3-7f62-413b-a1a3-9f98c6a23ab8 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:50.554046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-bf9519d3-7f62-413b-a1a3-9f98c6a23ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:50.557106 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1243/2456] 10.222.0.46 () {58 vars in 1058 bytes} [Fri Sep 9 13:02:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:50.578013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddc91d-616f-4d36-9597-d728561d57e4 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:50.583263 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9519d3-7f62-413b-a1a3-9f98c6a23ab8 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:50.588098 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420902e-3580-425d-8030-ac031a6a1b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:50.588332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420902e-3580-425d-8030-ac031a6a1b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:50.588612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420902e-3580-425d-8030-ac031a6a1b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:50.588882 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420902e-3580-425d-8030-ac031a6a1b7c admin admin] PATH_INFO: `/v3/users/48f14caeb8f944c6aaf27913972869c7` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:50.590213 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420902e-3580-425d-8030-ac031a6a1b7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=48f14caeb8f944c6aaf27913972869c7)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:50.590994 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddc91d-616f-4d36-9597-d728561d57e4 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:50.592279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420902e-3580-425d-8030-ac031a6a1b7c admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:50.597001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e3534-af88-418d-9b40-aa355bfcadaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:50.597765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e3534-af88-418d-9b40-aa355bfcadaa service placement] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:50.598112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e3534-af88-418d-9b40-aa355bfcadaa service placement] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:50.598512 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e3534-af88-418d-9b40-aa355bfcadaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:50.600346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a39e3534-af88-418d-9b40-aa355bfcadaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:50.613253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e3534-af88-418d-9b40-aa355bfcadaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:50.614870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e3534-af88-418d-9b40-aa355bfcadaa service placement] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:50.631001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1244/2457] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:02:50.754440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c420902e-3580-425d-8030-ac031a6a1b7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48f14caeb8f944c6aaf27913972869c7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:50.826985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c420902e-3580-425d-8030-ac031a6a1b7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48f14caeb8f944c6aaf27913972869c7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:50.827620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c420902e-3580-425d-8030-ac031a6a1b7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48f14caeb8f944c6aaf27913972869c7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:50.828335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c420902e-3580-425d-8030-ac031a6a1b7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48f14caeb8f944c6aaf27913972869c7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:50.836496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c420902e-3580-425d-8030-ac031a6a1b7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48f14caeb8f944c6aaf27913972869c7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:50.856133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1214/2458] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:02:50 2022] DELETE /identity/v3/users/48f14caeb8f944c6aaf27913972869c7 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:51.089956 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e3534-af88-418d-9b40-aa355bfcadaa service placement] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:51.131459 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:51.145087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e3534-af88-418d-9b40-aa355bfcadaa service placement] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:51.201159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54908e40-e643-4ec4-b717-f14ac3feee4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:51.201784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54908e40-e643-4ec4-b717-f14ac3feee4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.202059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54908e40-e643-4ec4-b717-f14ac3feee4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.202352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54908e40-e643-4ec4-b717-f14ac3feee4e admin admin] PATH_INFO: `/v3/projects/bf202f6eebd442e988e0f7307950a6c8` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.203499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54908e40-e643-4ec4-b717-f14ac3feee4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf202f6eebd442e988e0f7307950a6c8)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:51.205368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54908e40-e643-4ec4-b717-f14ac3feee4e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:51.319728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-54908e40-e643-4ec4-b717-f14ac3feee4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf202f6eebd442e988e0f7307950a6c8', 'request_id': 'req-54908e40-e643-4ec4-b717-f14ac3feee4e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:51.326770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-54908e40-e643-4ec4-b717-f14ac3feee4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf202f6eebd442e988e0f7307950a6c8', 'request_id': 'req-54908e40-e643-4ec4-b717-f14ac3feee4e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:51.404180 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-54908e40-e643-4ec4-b717-f14ac3feee4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf202f6eebd442e988e0f7307950a6c8', 'request_id': 'req-54908e40-e643-4ec4-b717-f14ac3feee4e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:51.428447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-54908e40-e643-4ec4-b717-f14ac3feee4e admin admin] The token cache is being invalidate because project bf202f6eebd442e988e0f7307950a6c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:02:51.428740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-54908e40-e643-4ec4-b717-f14ac3feee4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:51.429372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-54908e40-e643-4ec4-b717-f14ac3feee4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:51.430913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1245/2459] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:02:51 2022] DELETE /identity/v3/projects/bf202f6eebd442e988e0f7307950a6c8 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:51.436209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe53c84-d6fd-4601-9eef-96a2ab8d8f86 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.436704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe53c84-d6fd-4601-9eef-96a2ab8d8f86 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.436842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe53c84-d6fd-4601-9eef-96a2ab8d8f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.503751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ffe53c84-d6fd-4601-9eef-96a2ab8d8f86 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:51.504865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ffe53c84-d6fd-4601-9eef-96a2ab8d8f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:51.562129 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1215/2460] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:51.570925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedd39f-651f-4013-ba65-5396907e4cef None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.571432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedd39f-651f-4013-ba65-5396907e4cef None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.571538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedd39f-651f-4013-ba65-5396907e4cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.591039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-8aedd39f-651f-4013-ba65-5396907e4cef None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:51.592233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8aedd39f-651f-4013-ba65-5396907e4cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:51.599958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1246/2461] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:51.604104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe53c84-d6fd-4601-9eef-96a2ab8d8f86 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:51.614548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe53c84-d6fd-4601-9eef-96a2ab8d8f86 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:51.619064 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805c0be-07c3-4ca9-bb0e-f785ddc5623d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:51.619658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805c0be-07c3-4ca9-bb0e-f785ddc5623d admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.619938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805c0be-07c3-4ca9-bb0e-f785ddc5623d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.620209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805c0be-07c3-4ca9-bb0e-f785ddc5623d admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.621432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805c0be-07c3-4ca9-bb0e-f785ddc5623d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:51.621813 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805c0be-07c3-4ca9-bb0e-f785ddc5623d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:51.623419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805c0be-07c3-4ca9-bb0e-f785ddc5623d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:51.634635 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1216/2462] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:51.638783 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aedd39f-651f-4013-ba65-5396907e4cef None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:51.647732 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aedd39f-651f-4013-ba65-5396907e4cef None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:51.652502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efe669-1ec4-4b74-bc41-e75e9bbc624a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:51.653083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efe669-1ec4-4b74-bc41-e75e9bbc624a admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.653363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efe669-1ec4-4b74-bc41-e75e9bbc624a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.653636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efe669-1ec4-4b74-bc41-e75e9bbc624a admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.654266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efe669-1ec4-4b74-bc41-e75e9bbc624a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:51.656271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efe669-1ec4-4b74-bc41-e75e9bbc624a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:51.737082 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1247/2463] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:02:51 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:51.741272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805c0be-07c3-4ca9-bb0e-f785ddc5623d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:51.751835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805c0be-07c3-4ca9-bb0e-f785ddc5623d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:51.770739 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:51.816613 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e05e69-74ad-4ec0-9090-c91df8bd1405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:51.821015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e05e69-74ad-4ec0-9090-c91df8bd1405 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.821015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e05e69-74ad-4ec0-9090-c91df8bd1405 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.821015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e05e69-74ad-4ec0-9090-c91df8bd1405 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.821015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e05e69-74ad-4ec0-9090-c91df8bd1405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:51.821015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e05e69-74ad-4ec0-9090-c91df8bd1405 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:51.894116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1217/2464] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:51 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:51.898338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efe669-1ec4-4b74-bc41-e75e9bbc624a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:51.910449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efe669-1ec4-4b74-bc41-e75e9bbc624a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:51.915540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8a4bf-39e6-4a17-9f5d-03af7511249b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:51.926307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8a4bf-39e6-4a17-9f5d-03af7511249b admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.926307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8a4bf-39e6-4a17-9f5d-03af7511249b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.926307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8a4bf-39e6-4a17-9f5d-03af7511249b admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.926307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8a4bf-39e6-4a17-9f5d-03af7511249b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:51.926307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8a4bf-39e6-4a17-9f5d-03af7511249b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:51.935494 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1248/2465] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:02:51.939185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e05e69-74ad-4ec0-9090-c91df8bd1405 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:51.951855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e05e69-74ad-4ec0-9090-c91df8bd1405 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:51.957522 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8c69d-ac71-44ca-8f56-5a757b45544a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:51.958120 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c69d-ac71-44ca-8f56-5a757b45544a admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:51.958518 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c69d-ac71-44ca-8f56-5a757b45544a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:51.959588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c69d-ac71-44ca-8f56-5a757b45544a admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:51.959588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8c69d-ac71-44ca-8f56-5a757b45544a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:51.961272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8c69d-ac71-44ca-8f56-5a757b45544a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:51.981486 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1218/2466] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:02:51.982700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8a4bf-39e6-4a17-9f5d-03af7511249b admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:51.991720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8a4bf-39e6-4a17-9f5d-03af7511249b admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:51.998920 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a85641-f893-4d33-81b4-b8e203fa95ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:51.999659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a85641-f893-4d33-81b4-b8e203fa95ca admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.000004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a85641-f893-4d33-81b4-b8e203fa95ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.000342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a85641-f893-4d33-81b4-b8e203fa95ca admin admin] PATH_INFO: `/v3/projects/dc64ed3e7010468c98f4eb60b63de89f/users/2b35c40b8af4424cbfdb11f56f084bda/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.020187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a85641-f893-4d33-81b4-b8e203fa95ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc64ed3e7010468c98f4eb60b63de89f, user_id=2b35c40b8af4424cbfdb11f56f084bda, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:52.022953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a85641-f893-4d33-81b4-b8e203fa95ca admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:52.094840 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:52.099225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1249/2467] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:02:51 2022] PUT /identity/v3/projects/dc64ed3e7010468c98f4eb60b63de89f/users/2b35c40b8af4424cbfdb11f56f084bda/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:52.109503 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267635f-3425-4375-831e-c6978455c5cd None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.109810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267635f-3425-4375-831e-c6978455c5cd None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.110087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267635f-3425-4375-831e-c6978455c5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.141878 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-c267635f-3425-4375-831e-c6978455c5cd None None] MFA Rules not processed for user `2b35c40b8af4424cbfdb11f56f084bda`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:52.143338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c267635f-3425-4375-831e-c6978455c5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:52.221657 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1219/2468] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:52.225767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a85641-f893-4d33-81b4-b8e203fa95ca admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:52.314060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a85641-f893-4d33-81b4-b8e203fa95ca admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:52.319807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d197bb-dbe1-4071-97d1-e01db21a5424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:52.320414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d197bb-dbe1-4071-97d1-e01db21a5424 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.320805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d197bb-dbe1-4071-97d1-e01db21a5424 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.321083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d197bb-dbe1-4071-97d1-e01db21a5424 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.321710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d197bb-dbe1-4071-97d1-e01db21a5424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:52.323641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d197bb-dbe1-4071-97d1-e01db21a5424 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:52.419190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1250/2469] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:02:52 2022] POST /identity/v3/projects => generated 379 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:52.423641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267635f-3425-4375-831e-c6978455c5cd None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:52.435523 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267635f-3425-4375-831e-c6978455c5cd None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:52.504286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74043fea-23bd-4cfe-b31b-84d5f6ac2300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:52.505572 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74043fea-23bd-4cfe-b31b-84d5f6ac2300 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.505857 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74043fea-23bd-4cfe-b31b-84d5f6ac2300 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.506134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74043fea-23bd-4cfe-b31b-84d5f6ac2300 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.506824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74043fea-23bd-4cfe-b31b-84d5f6ac2300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:52.508735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74043fea-23bd-4cfe-b31b-84d5f6ac2300 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:52.599722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1220/2470] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:52 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:02:52.603744 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d197bb-dbe1-4071-97d1-e01db21a5424 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:52.615095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d197bb-dbe1-4071-97d1-e01db21a5424 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:52.619870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf8789-7062-490b-98c9-2eb29456f16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:52.620455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf8789-7062-490b-98c9-2eb29456f16a admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.620733 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf8789-7062-490b-98c9-2eb29456f16a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.621775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf8789-7062-490b-98c9-2eb29456f16a admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.622748 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf8789-7062-490b-98c9-2eb29456f16a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:52.624323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf8789-7062-490b-98c9-2eb29456f16a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:52.634750 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1251/2471] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:02:52.640488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74043fea-23bd-4cfe-b31b-84d5f6ac2300 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:52.651937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74043fea-23bd-4cfe-b31b-84d5f6ac2300 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:52.657304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e85109-b55b-43dc-a006-92ccbd15e959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:52.661110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e85109-b55b-43dc-a006-92ccbd15e959 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.661110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e85109-b55b-43dc-a006-92ccbd15e959 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.661110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e85109-b55b-43dc-a006-92ccbd15e959 admin admin] PATH_INFO: `/v3/projects/2c353237e3fd4effa1edfe897ecd2417/users/47f3ca237dd2484b844116b25f3823b3/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.674512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e85109-b55b-43dc-a006-92ccbd15e959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c353237e3fd4effa1edfe897ecd2417, user_id=47f3ca237dd2484b844116b25f3823b3, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:52.677134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e85109-b55b-43dc-a006-92ccbd15e959 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:52.679366 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:52.770763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1221/2472] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:02:52 2022] PUT /identity/v3/projects/2c353237e3fd4effa1edfe897ecd2417/users/47f3ca237dd2484b844116b25f3823b3/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:52.775607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf8789-7062-490b-98c9-2eb29456f16a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:52.788261 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf8789-7062-490b-98c9-2eb29456f16a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:52.844974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a46095-e280-453f-9a34-8e6168dbb7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:52.845578 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a46095-e280-453f-9a34-8e6168dbb7b8 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.845894 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a46095-e280-453f-9a34-8e6168dbb7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.846166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a46095-e280-453f-9a34-8e6168dbb7b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.846896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a46095-e280-453f-9a34-8e6168dbb7b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:52.848488 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a46095-e280-453f-9a34-8e6168dbb7b8 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:52.858885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1252/2473] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:02:52.863052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e85109-b55b-43dc-a006-92ccbd15e959 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:52.873459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e85109-b55b-43dc-a006-92ccbd15e959 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:52.878387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871e3d2-a704-49a3-9bdc-069256925d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:52.879097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871e3d2-a704-49a3-9bdc-069256925d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.882603 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871e3d2-a704-49a3-9bdc-069256925d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.882603 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871e3d2-a704-49a3-9bdc-069256925d29 admin admin] PATH_INFO: `/v3/projects/2c353237e3fd4effa1edfe897ecd2417/users/47f3ca237dd2484b844116b25f3823b3/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.882603 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871e3d2-a704-49a3-9bdc-069256925d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c353237e3fd4effa1edfe897ecd2417, user_id=47f3ca237dd2484b844116b25f3823b3, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:52.884778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871e3d2-a704-49a3-9bdc-069256925d29 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:52.896629 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1222/2474] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:02:52 2022] PUT /identity/v3/projects/2c353237e3fd4effa1edfe897ecd2417/users/47f3ca237dd2484b844116b25f3823b3/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:52.907142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ade789-3ece-470d-8782-3e1dabf2e321 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:52.907365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ade789-3ece-470d-8782-3e1dabf2e321 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:52.907595 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ade789-3ece-470d-8782-3e1dabf2e321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:52.936779 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-82ade789-3ece-470d-8782-3e1dabf2e321 None None] MFA Rules not processed for user `47f3ca237dd2484b844116b25f3823b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:52.938382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-82ade789-3ece-470d-8782-3e1dabf2e321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:53.009136 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:53.018526 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1253/2475] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:52 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:53.031907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871e3d2-a704-49a3-9bdc-069256925d29 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:53.033661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0871e3d2-a704-49a3-9bdc-069256925d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:53.131499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871e3d2-a704-49a3-9bdc-069256925d29 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:53.135543 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee4a61-da4c-4f24-a653-17cb82334b81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:53.136153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee4a61-da4c-4f24-a653-17cb82334b81 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:53.136431 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee4a61-da4c-4f24-a653-17cb82334b81 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:53.136789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee4a61-da4c-4f24-a653-17cb82334b81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:53.146053 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee4a61-da4c-4f24-a653-17cb82334b81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:53.154685 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee4a61-da4c-4f24-a653-17cb82334b81 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:53.157375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1223/2476] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:02:53.272410 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ade789-3ece-470d-8782-3e1dabf2e321 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:53.282702 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ade789-3ece-470d-8782-3e1dabf2e321 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:53.286280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efacc4-3095-4019-97ef-c40da4692d8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:53.286911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efacc4-3095-4019-97ef-c40da4692d8b service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:53.287191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efacc4-3095-4019-97ef-c40da4692d8b service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:53.287464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efacc4-3095-4019-97ef-c40da4692d8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:53.349598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72efacc4-3095-4019-97ef-c40da4692d8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:53.351317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72efacc4-3095-4019-97ef-c40da4692d8b service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:53.362080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1254/2477] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:02:53.419359 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee4a61-da4c-4f24-a653-17cb82334b81 service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:53.420967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-68ee4a61-da4c-4f24-a653-17cb82334b81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:53.509599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee4a61-da4c-4f24-a653-17cb82334b81 service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:53.514505 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e6b8d-1266-49b4-a1ea-209e3f36c3ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:53.515111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e6b8d-1266-49b4-a1ea-209e3f36c3ac service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:53.515389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e6b8d-1266-49b4-a1ea-209e3f36c3ac service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:53.516225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e6b8d-1266-49b4-a1ea-209e3f36c3ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:53.518681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-661e6b8d-1266-49b4-a1ea-209e3f36c3ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:53.564236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:53.585537 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e6b8d-1266-49b4-a1ea-209e3f36c3ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:53.587012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e6b8d-1266-49b4-a1ea-209e3f36c3ac service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:53.597380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1224/2478] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:02:56.556002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efacc4-3095-4019-97ef-c40da4692d8b service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:56.568609 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efacc4-3095-4019-97ef-c40da4692d8b service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:56.573261 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:56.573888 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:56.574856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:56.575135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] PATH_INFO: `/v3/users/2b35c40b8af4424cbfdb11f56f084bda` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:56.576288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b35c40b8af4424cbfdb11f56f084bda)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:56.578199 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:56.848664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b35c40b8af4424cbfdb11f56f084bda'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:56.947206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b35c40b8af4424cbfdb11f56f084bda'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:56.950694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b35c40b8af4424cbfdb11f56f084bda'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:56.951518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b35c40b8af4424cbfdb11f56f084bda'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:56.972448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b35c40b8af4424cbfdb11f56f084bda'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:56.990676 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1255/2479] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:02:56 2022] DELETE /identity/v3/users/2b35c40b8af4424cbfdb11f56f084bda => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:56.994330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11958a-98cd-48d5-8785-81bfdbafdc13 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:56.995147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11958a-98cd-48d5-8785-81bfdbafdc13 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:56.995147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11958a-98cd-48d5-8785-81bfdbafdc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:57.015342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-0f11958a-98cd-48d5-8785-81bfdbafdc13 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:57.016576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0f11958a-98cd-48d5-8785-81bfdbafdc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:57.095496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1225/2480] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:02:57.106343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:57.107991 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:57.194144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a7885-327d-4613-927d-bd05d46dfd21 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:57.198225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b4850-6c92-4209-b3f0-b3f38676bfeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:57.199097 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b4850-6c92-4209-b3f0-b3f38676bfeb service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:57.199448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b4850-6c92-4209-b3f0-b3f38676bfeb service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:57.199805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b4850-6c92-4209-b3f0-b3f38676bfeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:57.210497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b4850-6c92-4209-b3f0-b3f38676bfeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:57.212039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b4850-6c92-4209-b3f0-b3f38676bfeb service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:57.224591 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1256/2481] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:02:58.051549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11958a-98cd-48d5-8785-81bfdbafdc13 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:58.053001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0f11958a-98cd-48d5-8785-81bfdbafdc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:58.065527 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11958a-98cd-48d5-8785-81bfdbafdc13 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:58.071233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:58.071689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:58.072499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:58.073004 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] PATH_INFO: `/v3/projects/dc64ed3e7010468c98f4eb60b63de89f` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:58.074697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc64ed3e7010468c98f4eb60b63de89f)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:58.076995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:58.133508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b4850-6c92-4209-b3f0-b3f38676bfeb service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:58.134997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-486b4850-6c92-4209-b3f0-b3f38676bfeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:58.160215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b4850-6c92-4209-b3f0-b3f38676bfeb service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:58.165699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:58.166411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:58.166847 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:58.167368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] PATH_INFO: `/v3/users/63a2902f465f4fd1aed80060c2669259` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:58.168983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a2902f465f4fd1aed80060c2669259)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:58.171160 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:58.173829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:58.213259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc64ed3e7010468c98f4eb60b63de89f', 'request_id': 'req-ea8681c2-e916-49e3-9ab9-cb7dc162974b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.220373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc64ed3e7010468c98f4eb60b63de89f', 'request_id': 'req-ea8681c2-e916-49e3-9ab9-cb7dc162974b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.280686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc64ed3e7010468c98f4eb60b63de89f', 'request_id': 'req-ea8681c2-e916-49e3-9ab9-cb7dc162974b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.293277 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] The token cache is being invalidate because project dc64ed3e7010468c98f4eb60b63de89f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:02:58.294315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.295070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.296744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1226/2482] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:02:58 2022] DELETE /identity/v3/projects/dc64ed3e7010468c98f4eb60b63de89f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:58.300717 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:58.302562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:58.373054 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8681c2-e916-49e3-9ab9-cb7dc162974b admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:58.373665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a2902f465f4fd1aed80060c2669259'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.378733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:58.379409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:58.379800 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:58.380065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] PATH_INFO: `/v3/users/47f3ca237dd2484b844116b25f3823b3` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:58.381220 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f3ca237dd2484b844116b25f3823b3)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:58.383200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:58.385740 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:58.460829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a2902f465f4fd1aed80060c2669259'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.461573 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a2902f465f4fd1aed80060c2669259'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.462206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a2902f465f4fd1aed80060c2669259'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.476132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a2902f465f4fd1aed80060c2669259'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.492475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1257/2483] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:02:58 2022] DELETE /identity/v3/users/63a2902f465f4fd1aed80060c2669259 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:02:58.582985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f3ca237dd2484b844116b25f3823b3'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.657581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f3ca237dd2484b844116b25f3823b3'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.658311 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f3ca237dd2484b844116b25f3823b3'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.659088 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f3ca237dd2484b844116b25f3823b3'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.667373 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f3ca237dd2484b844116b25f3823b3'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:58.692949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1227/2484] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:02:58 2022] DELETE /identity/v3/users/47f3ca237dd2484b844116b25f3823b3 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:02:58.756695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:58.758905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:58.858763 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89b0c7-f0be-4da5-acb7-2b1aed158e3f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:58.922210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c765ab-9a7b-484e-8e04-6bbdf001de36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:58.922210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c765ab-9a7b-484e-8e04-6bbdf001de36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:58.922210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c765ab-9a7b-484e-8e04-6bbdf001de36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:58.922210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c765ab-9a7b-484e-8e04-6bbdf001de36 admin admin] PATH_INFO: `/v3/projects/ceed6a1f0f4542cbafa4b2a46b16d5fb` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:58.923429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c765ab-9a7b-484e-8e04-6bbdf001de36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceed6a1f0f4542cbafa4b2a46b16d5fb)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:58.925313 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c765ab-9a7b-484e-8e04-6bbdf001de36 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:59.051947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-02c765ab-9a7b-484e-8e04-6bbdf001de36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceed6a1f0f4542cbafa4b2a46b16d5fb', 'request_id': 'req-02c765ab-9a7b-484e-8e04-6bbdf001de36'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:59.058015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-02c765ab-9a7b-484e-8e04-6bbdf001de36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceed6a1f0f4542cbafa4b2a46b16d5fb', 'request_id': 'req-02c765ab-9a7b-484e-8e04-6bbdf001de36'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:59.153063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-02c765ab-9a7b-484e-8e04-6bbdf001de36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceed6a1f0f4542cbafa4b2a46b16d5fb', 'request_id': 'req-02c765ab-9a7b-484e-8e04-6bbdf001de36'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:59.160362 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:59.162918 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:59.166416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-02c765ab-9a7b-484e-8e04-6bbdf001de36 admin admin] The token cache is being invalidate because project ceed6a1f0f4542cbafa4b2a46b16d5fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:02:59.168932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-02c765ab-9a7b-484e-8e04-6bbdf001de36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:59.168932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-02c765ab-9a7b-484e-8e04-6bbdf001de36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:59.169362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1258/2485] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:02:58 2022] DELETE /identity/v3/projects/ceed6a1f0f4542cbafa4b2a46b16d5fb => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:02:59.174662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb845d-532a-47ac-8f7c-ba4e47b8d771 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.174982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb845d-532a-47ac-8f7c-ba4e47b8d771 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.175330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb845d-532a-47ac-8f7c-ba4e47b8d771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.177849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240b636-bc98-4013-b071-f6ba184dfead admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:59.196735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-fedb845d-532a-47ac-8f7c-ba4e47b8d771 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:59.203615 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fedb845d-532a-47ac-8f7c-ba4e47b8d771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:59.240142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:59.240760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.241038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.241313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d admin admin] PATH_INFO: `/v3/projects/2c353237e3fd4effa1edfe897ecd2417` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.242470 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c353237e3fd4effa1edfe897ecd2417)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:59.246322 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:59.263550 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1259/2486] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:02:59.270702 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67217b9-72e1-46fe-8089-9932ac1559f7 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.270995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67217b9-72e1-46fe-8089-9932ac1559f7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.271273 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67217b9-72e1-46fe-8089-9932ac1559f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.292012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-f67217b9-72e1-46fe-8089-9932ac1559f7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:59.293791 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f67217b9-72e1-46fe-8089-9932ac1559f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:59.302960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1260/2487] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:02:59.306949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67217b9-72e1-46fe-8089-9932ac1559f7 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:59.317437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67217b9-72e1-46fe-8089-9932ac1559f7 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:59.322563 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225917f-7198-49f2-b680-4aca244c0f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:59.323057 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225917f-7198-49f2-b680-4aca244c0f9f admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.323344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225917f-7198-49f2-b680-4aca244c0f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.323618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225917f-7198-49f2-b680-4aca244c0f9f admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.324363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225917f-7198-49f2-b680-4aca244c0f9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:59.324662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225917f-7198-49f2-b680-4aca244c0f9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:59.326235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225917f-7198-49f2-b680-4aca244c0f9f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:59.339458 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1261/2488] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:02:59.343832 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225917f-7198-49f2-b680-4aca244c0f9f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:59.353704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225917f-7198-49f2-b680-4aca244c0f9f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:59.365194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c353237e3fd4effa1edfe897ecd2417', 'request_id': 'req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:59.371669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c353237e3fd4effa1edfe897ecd2417', 'request_id': 'req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:59.373497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:59.413101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4af66-a490-4b36-9a43-a4986fa3772d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:59.413803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4af66-a490-4b36-9a43-a4986fa3772d admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.414081 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4af66-a490-4b36-9a43-a4986fa3772d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.414349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4af66-a490-4b36-9a43-a4986fa3772d admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.415086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4af66-a490-4b36-9a43-a4986fa3772d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:59.417139 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4af66-a490-4b36-9a43-a4986fa3772d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:59.420784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c353237e3fd4effa1edfe897ecd2417', 'request_id': 'req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:59.431809 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d admin admin] The token cache is being invalidate because project 2c353237e3fd4effa1edfe897ecd2417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:02:59.432021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:59.432667 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2a7d4baf-b72b-4118-8bd5-9fac58bbc73d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:02:59.435132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1228/2489] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:02:59 2022] DELETE /identity/v3/projects/2c353237e3fd4effa1edfe897ecd2417 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:02:59.439776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e9a97-0d54-407b-8119-729233e0f860 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.440119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e9a97-0d54-407b-8119-729233e0f860 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.440398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e9a97-0d54-407b-8119-729233e0f860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.460612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-e68e9a97-0d54-407b-8119-729233e0f860 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:59.461796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e68e9a97-0d54-407b-8119-729233e0f860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:59.469677 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1229/2490] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:02:59.477132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016e5e0-87db-4de2-b98b-1cdb4425cb09 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.477487 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016e5e0-87db-4de2-b98b-1cdb4425cb09 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.477752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016e5e0-87db-4de2-b98b-1cdb4425cb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.485227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1262/2491] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:02:59 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:02:59.489597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4af66-a490-4b36-9a43-a4986fa3772d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:59.492060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-45e4af66-a490-4b36-9a43-a4986fa3772d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:59.497630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-5016e5e0-87db-4de2-b98b-1cdb4425cb09 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:59.498829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5016e5e0-87db-4de2-b98b-1cdb4425cb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:59.540938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:02:59.555445 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4af66-a490-4b36-9a43-a4986fa3772d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:59.560061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1230/2492] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:02:59.560448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ff362-ca27-4c02-bb4b-f51c0ca05d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:59.561087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ff362-ca27-4c02-bb4b-f51c0ca05d06 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.561364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ff362-ca27-4c02-bb4b-f51c0ca05d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.561841 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ff362-ca27-4c02-bb4b-f51c0ca05d06 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.562645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ff362-ca27-4c02-bb4b-f51c0ca05d06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:59.564398 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016e5e0-87db-4de2-b98b-1cdb4425cb09 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:59.564660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ff362-ca27-4c02-bb4b-f51c0ca05d06 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:59.573205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016e5e0-87db-4de2-b98b-1cdb4425cb09 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:59.578227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4b438-274d-4b1b-925d-9e1fff822f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:59.578858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4b438-274d-4b1b-925d-9e1fff822f6a admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.579182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4b438-274d-4b1b-925d-9e1fff822f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.579459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4b438-274d-4b1b-925d-9e1fff822f6a admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.580093 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4b438-274d-4b1b-925d-9e1fff822f6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:02:59.580394 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4b438-274d-4b1b-925d-9e1fff822f6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:59.582290 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4b438-274d-4b1b-925d-9e1fff822f6a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:59.594323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1231/2493] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:02:59.598502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4b438-274d-4b1b-925d-9e1fff822f6a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:59.608119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4b438-274d-4b1b-925d-9e1fff822f6a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:59.612760 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd84fa9-071f-4e08-9cbf-dada51ac61d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:59.613331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd84fa9-071f-4e08-9cbf-dada51ac61d0 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.613605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd84fa9-071f-4e08-9cbf-dada51ac61d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.613883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd84fa9-071f-4e08-9cbf-dada51ac61d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.614507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd84fa9-071f-4e08-9cbf-dada51ac61d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:59.616443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd84fa9-071f-4e08-9cbf-dada51ac61d0 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:59.660797 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1263/2494] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:59 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:02:59.665308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ff362-ca27-4c02-bb4b-f51c0ca05d06 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:59.675118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ff362-ca27-4c02-bb4b-f51c0ca05d06 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:59.679710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd3902-c423-448a-91f1-400a3866ad3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:59.680291 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd3902-c423-448a-91f1-400a3866ad3d admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.680569 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd3902-c423-448a-91f1-400a3866ad3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.680840 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd3902-c423-448a-91f1-400a3866ad3d admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.681510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd3902-c423-448a-91f1-400a3866ad3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:59.693844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd3902-c423-448a-91f1-400a3866ad3d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:59.693844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1264/2495] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:02:59.697335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd3902-c423-448a-91f1-400a3866ad3d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:59.707514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd3902-c423-448a-91f1-400a3866ad3d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:59.712397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bb58b-bd74-4dcf-a125-d14fae82c57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:59.712969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bb58b-bd74-4dcf-a125-d14fae82c57c admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.713249 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bb58b-bd74-4dcf-a125-d14fae82c57c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.713521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bb58b-bd74-4dcf-a125-d14fae82c57c admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.714156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bb58b-bd74-4dcf-a125-d14fae82c57c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:59.715762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bb58b-bd74-4dcf-a125-d14fae82c57c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:59.726304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1265/2496] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:02:59.731974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bb58b-bd74-4dcf-a125-d14fae82c57c admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:59.741487 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bb58b-bd74-4dcf-a125-d14fae82c57c admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:59.746856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5df9c-8c77-4875-aa3d-9e2e16e4ce18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:59.748359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5df9c-8c77-4875-aa3d-9e2e16e4ce18 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.748359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5df9c-8c77-4875-aa3d-9e2e16e4ce18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.748359 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5df9c-8c77-4875-aa3d-9e2e16e4ce18 admin admin] PATH_INFO: `/v3/projects/4d8ac804454c4a268df09701c1c55ac2/users/8263eb1f157548519990dba4c2300e91/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.757189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1232/2497] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:02:59 2022] POST /identity/v3/projects => generated 365 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:02:59.758463 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd84fa9-071f-4e08-9cbf-dada51ac61d0 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:59.763260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5df9c-8c77-4875-aa3d-9e2e16e4ce18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d8ac804454c4a268df09701c1c55ac2, user_id=8263eb1f157548519990dba4c2300e91, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:59.768697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5df9c-8c77-4875-aa3d-9e2e16e4ce18 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:59.822280 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd84fa9-071f-4e08-9cbf-dada51ac61d0 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:59.827564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fc967-9c67-464e-936f-ebffa2d37e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:59.828160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fc967-9c67-464e-936f-ebffa2d37e9f admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.828560 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fc967-9c67-464e-936f-ebffa2d37e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.828837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fc967-9c67-464e-936f-ebffa2d37e9f admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.829484 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fc967-9c67-464e-936f-ebffa2d37e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:59.831714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fc967-9c67-464e-936f-ebffa2d37e9f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:59.844880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1266/2498] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:02:59 2022] PUT /identity/v3/projects/4d8ac804454c4a268df09701c1c55ac2/users/8263eb1f157548519990dba4c2300e91/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:02:59.857068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf4370-25cc-4a1e-8c4b-3a17370d7ce2 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.857362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf4370-25cc-4a1e-8c4b-3a17370d7ce2 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.857638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf4370-25cc-4a1e-8c4b-3a17370d7ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.885372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-7baf4370-25cc-4a1e-8c4b-3a17370d7ce2 None None] MFA Rules not processed for user `8263eb1f157548519990dba4c2300e91`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:02:59.886714 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7baf4370-25cc-4a1e-8c4b-3a17370d7ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:59.904605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1233/2499] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:02:59 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:02:59.905695 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fc967-9c67-464e-936f-ebffa2d37e9f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:59.915663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fc967-9c67-464e-936f-ebffa2d37e9f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:02:59.956147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1267/2500] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:02:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:02:59.963776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf4370-25cc-4a1e-8c4b-3a17370d7ce2 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:02:59.965511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7baf4370-25cc-4a1e-8c4b-3a17370d7ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:02:59.974232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835eca75-3989-46d1-b1aa-53368409ca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:02:59.974977 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eca75-3989-46d1-b1aa-53368409ca06 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:02:59.975310 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eca75-3989-46d1-b1aa-53368409ca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:02:59.975590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eca75-3989-46d1-b1aa-53368409ca06 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:02:59.982241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835eca75-3989-46d1-b1aa-53368409ca06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:02:59.982241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835eca75-3989-46d1-b1aa-53368409ca06 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:02:59.988508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1234/2501] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:02:59.992836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835eca75-3989-46d1-b1aa-53368409ca06 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:00.003478 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835eca75-3989-46d1-b1aa-53368409ca06 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:00.008193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627219b-7ce8-48ba-8edc-5ac1c0bed6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:00.016137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627219b-7ce8-48ba-8edc-5ac1c0bed6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.016137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627219b-7ce8-48ba-8edc-5ac1c0bed6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.016137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627219b-7ce8-48ba-8edc-5ac1c0bed6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.016137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627219b-7ce8-48ba-8edc-5ac1c0bed6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:00.016137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627219b-7ce8-48ba-8edc-5ac1c0bed6f9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:00.027355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1235/2502] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:03:00.028500 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627219b-7ce8-48ba-8edc-5ac1c0bed6f9 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:00.041006 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:00.043208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627219b-7ce8-48ba-8edc-5ac1c0bed6f9 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:00.050986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf4370-25cc-4a1e-8c4b-3a17370d7ce2 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:00.051117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb880e-c6b6-433e-9a55-5c41df42649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:00.051117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb880e-c6b6-433e-9a55-5c41df42649b admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.051117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb880e-c6b6-433e-9a55-5c41df42649b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.051117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb880e-c6b6-433e-9a55-5c41df42649b admin admin] PATH_INFO: `/v3/projects/6d993173df38456eaf2ecec027207fed/users/6b3e6d93921e43219e94c8908e651729/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.052788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746fa69-cea8-4375-9451-51a1680469f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:00.053400 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746fa69-cea8-4375-9451-51a1680469f0 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.053701 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746fa69-cea8-4375-9451-51a1680469f0 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.053979 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746fa69-cea8-4375-9451-51a1680469f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.063449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746fa69-cea8-4375-9451-51a1680469f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:00.064933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb880e-c6b6-433e-9a55-5c41df42649b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d993173df38456eaf2ecec027207fed, user_id=6b3e6d93921e43219e94c8908e651729, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:00.065054 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746fa69-cea8-4375-9451-51a1680469f0 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:00.067674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb880e-c6b6-433e-9a55-5c41df42649b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:00.073366 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:00.074916 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1268/2503] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:02:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:03:00.171403 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1236/2504] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:00 2022] PUT /identity/v3/projects/6d993173df38456eaf2ecec027207fed/users/6b3e6d93921e43219e94c8908e651729/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:03:00.181604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89d3d0-e826-4e3e-b932-c354b3692c44 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.182085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89d3d0-e826-4e3e-b932-c354b3692c44 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.182315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89d3d0-e826-4e3e-b932-c354b3692c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.215749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-ee89d3d0-e826-4e3e-b932-c354b3692c44 None None] MFA Rules not processed for user `6b3e6d93921e43219e94c8908e651729`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:00.216944 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ee89d3d0-e826-4e3e-b932-c354b3692c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:00.281762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1269/2505] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:00.286345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb880e-c6b6-433e-9a55-5c41df42649b admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:00.352223 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb880e-c6b6-433e-9a55-5c41df42649b admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:00.357236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3aa3c8-4db9-40e4-9089-2351cd828504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:00.357837 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aa3c8-4db9-40e4-9089-2351cd828504 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.358116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aa3c8-4db9-40e4-9089-2351cd828504 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.358384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aa3c8-4db9-40e4-9089-2351cd828504 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.359034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3aa3c8-4db9-40e4-9089-2351cd828504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:00.360903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3aa3c8-4db9-40e4-9089-2351cd828504 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:00.387808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:00.422951 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1237/2506] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:03:00 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:03:00.427422 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89d3d0-e826-4e3e-b932-c354b3692c44 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:00.438900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89d3d0-e826-4e3e-b932-c354b3692c44 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:00.498014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b33081-a38b-4700-af0c-a4d87b0c9cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:00.498650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b33081-a38b-4700-af0c-a4d87b0c9cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.498963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b33081-a38b-4700-af0c-a4d87b0c9cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.499238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b33081-a38b-4700-af0c-a4d87b0c9cd9 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.499871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b33081-a38b-4700-af0c-a4d87b0c9cd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:00.501751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b33081-a38b-4700-af0c-a4d87b0c9cd9 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:00.586414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1270/2507] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:03:00 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:03:00.591590 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3aa3c8-4db9-40e4-9089-2351cd828504 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:00.601981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3aa3c8-4db9-40e4-9089-2351cd828504 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:00.607219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec1f5a-f53d-4786-b3be-a931232445ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:00.607811 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec1f5a-f53d-4786-b3be-a931232445ad admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.608095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec1f5a-f53d-4786-b3be-a931232445ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.608369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec1f5a-f53d-4786-b3be-a931232445ad admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.609005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec1f5a-f53d-4786-b3be-a931232445ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:00.610656 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec1f5a-f53d-4786-b3be-a931232445ad admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:00.622556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1238/2508] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:00.626597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b33081-a38b-4700-af0c-a4d87b0c9cd9 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:00.637169 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b33081-a38b-4700-af0c-a4d87b0c9cd9 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:00.642035 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758995e-fba1-44c2-ab79-de6096950b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:00.642691 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758995e-fba1-44c2-ab79-de6096950b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.642968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758995e-fba1-44c2-ab79-de6096950b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.643237 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758995e-fba1-44c2-ab79-de6096950b3d admin admin] PATH_INFO: `/v3/projects/42a1c131dda542fcabe925a13472228b/users/63304e1105094ed88fa5584a9125903c/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.658594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758995e-fba1-44c2-ab79-de6096950b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a1c131dda542fcabe925a13472228b, user_id=63304e1105094ed88fa5584a9125903c, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:00.661204 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758995e-fba1-44c2-ab79-de6096950b3d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:00.775577 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1271/2509] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:00 2022] PUT /identity/v3/projects/42a1c131dda542fcabe925a13472228b/users/63304e1105094ed88fa5584a9125903c/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:00.780458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec1f5a-f53d-4786-b3be-a931232445ad admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:00.791006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec1f5a-f53d-4786-b3be-a931232445ad admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:00.849800 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd53e84-0adf-4ba3-a288-bd429d69e12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:00.850409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd53e84-0adf-4ba3-a288-bd429d69e12b admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.850726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd53e84-0adf-4ba3-a288-bd429d69e12b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.850998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd53e84-0adf-4ba3-a288-bd429d69e12b admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.851628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd53e84-0adf-4ba3-a288-bd429d69e12b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:00.853257 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd53e84-0adf-4ba3-a288-bd429d69e12b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:00.863436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1239/2510] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:00.869344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758995e-fba1-44c2-ab79-de6096950b3d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:00.879703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758995e-fba1-44c2-ab79-de6096950b3d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:00.884449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86885fa8-a116-4b49-a0fe-a09682c4f01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:00.885120 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86885fa8-a116-4b49-a0fe-a09682c4f01b admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.885402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86885fa8-a116-4b49-a0fe-a09682c4f01b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.885706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86885fa8-a116-4b49-a0fe-a09682c4f01b admin admin] PATH_INFO: `/v3/projects/42a1c131dda542fcabe925a13472228b/users/63304e1105094ed88fa5584a9125903c/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.888312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86885fa8-a116-4b49-a0fe-a09682c4f01b admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a1c131dda542fcabe925a13472228b, user_id=63304e1105094ed88fa5584a9125903c, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:00.891076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86885fa8-a116-4b49-a0fe-a09682c4f01b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:00.902636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1272/2511] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:00 2022] PUT /identity/v3/projects/42a1c131dda542fcabe925a13472228b/users/63304e1105094ed88fa5584a9125903c/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:00.914246 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1ce21-7e5e-41f3-9f32-94d2e8122f69 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:00.914586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1ce21-7e5e-41f3-9f32-94d2e8122f69 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:00.914866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1ce21-7e5e-41f3-9f32-94d2e8122f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:00.943115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-58a1ce21-7e5e-41f3-9f32-94d2e8122f69 None None] MFA Rules not processed for user `63304e1105094ed88fa5584a9125903c`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:00.944309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-58a1ce21-7e5e-41f3-9f32-94d2e8122f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:00.963086 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:01.018529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1240/2512] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:01.029948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86885fa8-a116-4b49-a0fe-a09682c4f01b admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:01.031661 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-86885fa8-a116-4b49-a0fe-a09682c4f01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:01.060073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:01.091592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86885fa8-a116-4b49-a0fe-a09682c4f01b admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:01.095243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa36cb-6f61-400e-bb69-42009869bf01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:01.095838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa36cb-6f61-400e-bb69-42009869bf01 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.096143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa36cb-6f61-400e-bb69-42009869bf01 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.096413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa36cb-6f61-400e-bb69-42009869bf01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.105587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa36cb-6f61-400e-bb69-42009869bf01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:01.107266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa36cb-6f61-400e-bb69-42009869bf01 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:01.117775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1273/2513] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:01.317383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1ce21-7e5e-41f3-9f32-94d2e8122f69 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:01.328329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1ce21-7e5e-41f3-9f32-94d2e8122f69 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:01.332160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adf460-87a7-44f0-ab56-46cb90551044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:01.332764 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adf460-87a7-44f0-ab56-46cb90551044 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:01.333045 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adf460-87a7-44f0-ab56-46cb90551044 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:01.333312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adf460-87a7-44f0-ab56-46cb90551044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:01.395673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43adf460-87a7-44f0-ab56-46cb90551044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:01.397180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43adf460-87a7-44f0-ab56-46cb90551044 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:01.406363 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1241/2514] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:07.624098 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa36cb-6f61-400e-bb69-42009869bf01 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:07.685548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa36cb-6f61-400e-bb69-42009869bf01 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:07.690191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:07.690855 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:07.691137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:07.691452 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] PATH_INFO: `/v3/users/6b3e6d93921e43219e94c8908e651729` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:07.692630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b3e6d93921e43219e94c8908e651729)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:07.694574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:07.810832 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b3e6d93921e43219e94c8908e651729'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:07.866566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b3e6d93921e43219e94c8908e651729'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:07.867223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b3e6d93921e43219e94c8908e651729'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:07.867831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b3e6d93921e43219e94c8908e651729'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:07.881264 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b3e6d93921e43219e94c8908e651729'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:07.894038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1274/2515] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:03:07 2022] DELETE /identity/v3/users/6b3e6d93921e43219e94c8908e651729 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:07.898600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7758aa-ee1d-4794-a47f-c5ce67856d90 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:07.899069 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7758aa-ee1d-4794-a47f-c5ce67856d90 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:07.899166 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7758aa-ee1d-4794-a47f-c5ce67856d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:07.918311 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-6a7758aa-ee1d-4794-a47f-c5ce67856d90 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:07.919771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6a7758aa-ee1d-4794-a47f-c5ce67856d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:07.977278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1242/2516] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:07.986951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:07.995202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:08.043947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:08.051022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198211d-079e-4a65-bb75-3a4cd2a7466f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:08.054812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122a680-ebec-43cc-8df6-56051deabd6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:08.055419 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122a680-ebec-43cc-8df6-56051deabd6d service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:08.055727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122a680-ebec-43cc-8df6-56051deabd6d service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:08.055999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122a680-ebec-43cc-8df6-56051deabd6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:08.068879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122a680-ebec-43cc-8df6-56051deabd6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:08.069340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122a680-ebec-43cc-8df6-56051deabd6d service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:08.080220 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1275/2517] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:08.175829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7758aa-ee1d-4794-a47f-c5ce67856d90 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:08.175829 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6a7758aa-ee1d-4794-a47f-c5ce67856d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:08.186681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7758aa-ee1d-4794-a47f-c5ce67856d90 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:08.193122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:08.193952 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:08.196530 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:08.197392 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] PATH_INFO: `/v3/users/8263eb1f157548519990dba4c2300e91` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:08.199459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] RBAC: Authorizing `identity:delete_user(user_id=8263eb1f157548519990dba4c2300e91)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:08.203082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:08.413249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8263eb1f157548519990dba4c2300e91'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:08.417943 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:08.469071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8263eb1f157548519990dba4c2300e91'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:08.470193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8263eb1f157548519990dba4c2300e91'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:08.470881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8263eb1f157548519990dba4c2300e91'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:08.481127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8263eb1f157548519990dba4c2300e91'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:08.504262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1243/2518] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:03:08 2022] DELETE /identity/v3/users/8263eb1f157548519990dba4c2300e91 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:08.510944 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef2e2-6c81-4191-befd-6b905d1b5982 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:08.511305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef2e2-6c81-4191-befd-6b905d1b5982 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:08.511742 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef2e2-6c81-4191-befd-6b905d1b5982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:08.536415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-5d9ef2e2-6c81-4191-befd-6b905d1b5982 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:08.537654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5d9ef2e2-6c81-4191-befd-6b905d1b5982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:08.599152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1276/2519] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:08.611744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:08.613219 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:08.616511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef2e2-6c81-4191-befd-6b905d1b5982 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:08.616953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5d9ef2e2-6c81-4191-befd-6b905d1b5982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:08.629686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9864a9-6f0f-4865-840b-0089edc771df admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:08.635151 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:08.635903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:08.636454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:08.636808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] PATH_INFO: `/v3/projects/6d993173df38456eaf2ecec027207fed` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:08.638805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d993173df38456eaf2ecec027207fed)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:08.641021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:08.689001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef2e2-6c81-4191-befd-6b905d1b5982 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:08.692743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a109e-0d21-4953-8052-94b0fef6636a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:08.693298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a109e-0d21-4953-8052-94b0fef6636a service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:08.697026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a109e-0d21-4953-8052-94b0fef6636a service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:08.697026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a109e-0d21-4953-8052-94b0fef6636a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:08.708570 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a109e-0d21-4953-8052-94b0fef6636a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:08.710172 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a109e-0d21-4953-8052-94b0fef6636a service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:08.721399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1277/2520] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:08.743312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d993173df38456eaf2ecec027207fed', 'request_id': 'req-9a4fe171-24dc-4a2a-99d6-e049148a8112'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:08.750234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d993173df38456eaf2ecec027207fed', 'request_id': 'req-9a4fe171-24dc-4a2a-99d6-e049148a8112'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:08.824691 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d993173df38456eaf2ecec027207fed', 'request_id': 'req-9a4fe171-24dc-4a2a-99d6-e049148a8112'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:08.837789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] The token cache is being invalidate because project 6d993173df38456eaf2ecec027207fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:03:08.838211 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:08.838925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:08.840781 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1244/2521] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:03:08 2022] DELETE /identity/v3/projects/6d993173df38456eaf2ecec027207fed => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:08.845293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a109e-0d21-4953-8052-94b0fef6636a service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:08.856179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a109e-0d21-4953-8052-94b0fef6636a service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:08.927828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783eacf9-f529-4b25-99c8-a3f6c867d5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:08.928468 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eacf9-f529-4b25-99c8-a3f6c867d5ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:08.928751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eacf9-f529-4b25-99c8-a3f6c867d5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:08.929022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783eacf9-f529-4b25-99c8-a3f6c867d5ea admin admin] PATH_INFO: `/v3/users/63304e1105094ed88fa5584a9125903c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:08.930202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783eacf9-f529-4b25-99c8-a3f6c867d5ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=63304e1105094ed88fa5584a9125903c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:08.932134 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783eacf9-f529-4b25-99c8-a3f6c867d5ea admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:09.096142 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:09.114537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-783eacf9-f529-4b25-99c8-a3f6c867d5ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63304e1105094ed88fa5584a9125903c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:09.180565 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-783eacf9-f529-4b25-99c8-a3f6c867d5ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63304e1105094ed88fa5584a9125903c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:09.181217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-783eacf9-f529-4b25-99c8-a3f6c867d5ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63304e1105094ed88fa5584a9125903c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:09.181923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-783eacf9-f529-4b25-99c8-a3f6c867d5ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63304e1105094ed88fa5584a9125903c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:09.195914 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:09.196440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-783eacf9-f529-4b25-99c8-a3f6c867d5ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63304e1105094ed88fa5584a9125903c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:09.199424 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:09.209935 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1278/2522] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:03:08 2022] DELETE /identity/v3/users/63304e1105094ed88fa5584a9125903c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:09.231220 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4fe171-24dc-4a2a-99d6-e049148a8112 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:09.317827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017919a0-aff8-444d-b4af-afa13e2fe961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:09.318879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017919a0-aff8-444d-b4af-afa13e2fe961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:09.319452 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017919a0-aff8-444d-b4af-afa13e2fe961 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:09.320586 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017919a0-aff8-444d-b4af-afa13e2fe961 admin admin] PATH_INFO: `/v3/projects/4d8ac804454c4a268df09701c1c55ac2` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:09.322301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017919a0-aff8-444d-b4af-afa13e2fe961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d8ac804454c4a268df09701c1c55ac2)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:09.326501 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017919a0-aff8-444d-b4af-afa13e2fe961 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:09.459627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-017919a0-aff8-444d-b4af-afa13e2fe961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d8ac804454c4a268df09701c1c55ac2', 'request_id': 'req-017919a0-aff8-444d-b4af-afa13e2fe961'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:09.462038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:09.470180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-017919a0-aff8-444d-b4af-afa13e2fe961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d8ac804454c4a268df09701c1c55ac2', 'request_id': 'req-017919a0-aff8-444d-b4af-afa13e2fe961'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:09.532232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-017919a0-aff8-444d-b4af-afa13e2fe961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d8ac804454c4a268df09701c1c55ac2', 'request_id': 'req-017919a0-aff8-444d-b4af-afa13e2fe961'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:09.599327 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-017919a0-aff8-444d-b4af-afa13e2fe961 admin admin] The token cache is being invalidate because project 4d8ac804454c4a268df09701c1c55ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:03:09.599614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-017919a0-aff8-444d-b4af-afa13e2fe961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:09.601844 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-017919a0-aff8-444d-b4af-afa13e2fe961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:09.610529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1245/2523] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:03:09 2022] DELETE /identity/v3/projects/4d8ac804454c4a268df09701c1c55ac2 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:03:09.612683 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bebd43-8308-4f23-9b35-14b49cb7f22c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:09.612963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bebd43-8308-4f23-9b35-14b49cb7f22c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:09.613395 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bebd43-8308-4f23-9b35-14b49cb7f22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:09.650411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-06bebd43-8308-4f23-9b35-14b49cb7f22c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:09.656814 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-06bebd43-8308-4f23-9b35-14b49cb7f22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:09.757242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1279/2524] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:09.758132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bb34f-adca-45a8-96b1-e40aa0d460d4 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:09.758419 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bb34f-adca-45a8-96b1-e40aa0d460d4 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:09.763747 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bb34f-adca-45a8-96b1-e40aa0d460d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:09.806731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-326bb34f-adca-45a8-96b1-e40aa0d460d4 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:09.808714 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-326bb34f-adca-45a8-96b1-e40aa0d460d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:09.823411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1246/2525] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:09.828684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bebd43-8308-4f23-9b35-14b49cb7f22c None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:09.842321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bebd43-8308-4f23-9b35-14b49cb7f22c None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:09.847854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec50915-f351-406d-9574-be268e118ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:09.848989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec50915-f351-406d-9574-be268e118ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:09.849568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec50915-f351-406d-9574-be268e118ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:09.850007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec50915-f351-406d-9574-be268e118ad1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:09.851007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec50915-f351-406d-9574-be268e118ad1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:09.851462 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec50915-f351-406d-9574-be268e118ad1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:09.853255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec50915-f351-406d-9574-be268e118ad1 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:09.866250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bb34f-adca-45a8-96b1-e40aa0d460d4 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:09.868101 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-326bb34f-adca-45a8-96b1-e40aa0d460d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:09.882425 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bb34f-adca-45a8-96b1-e40aa0d460d4 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:09.885970 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1280/2526] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:09.891489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec50915-f351-406d-9574-be268e118ad1 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:09.891785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e3f7-8d42-4a93-a73a-6075152fffc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:09.891785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e3f7-8d42-4a93-a73a-6075152fffc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:09.891785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e3f7-8d42-4a93-a73a-6075152fffc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:09.891785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e3f7-8d42-4a93-a73a-6075152fffc1 admin admin] PATH_INFO: `/v3/projects/42a1c131dda542fcabe925a13472228b` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:09.893013 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12e3f7-8d42-4a93-a73a-6075152fffc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a1c131dda542fcabe925a13472228b)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:09.895176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12e3f7-8d42-4a93-a73a-6075152fffc1 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:09.898144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec50915-f351-406d-9574-be268e118ad1 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:09.905445 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7228d-afaf-4d2d-92f6-6fb6f5bdd603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:09.907138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7228d-afaf-4d2d-92f6-6fb6f5bdd603 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:09.907350 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7228d-afaf-4d2d-92f6-6fb6f5bdd603 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:09.908222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7228d-afaf-4d2d-92f6-6fb6f5bdd603 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:09.908887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7228d-afaf-4d2d-92f6-6fb6f5bdd603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:09.914372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7228d-afaf-4d2d-92f6-6fb6f5bdd603 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:09.992510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1281/2527] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:03:09 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:09.993696 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7228d-afaf-4d2d-92f6-6fb6f5bdd603 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:10.002704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-ac12e3f7-8d42-4a93-a73a-6075152fffc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a1c131dda542fcabe925a13472228b', 'request_id': 'req-ac12e3f7-8d42-4a93-a73a-6075152fffc1'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:10.012222 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-ac12e3f7-8d42-4a93-a73a-6075152fffc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a1c131dda542fcabe925a13472228b', 'request_id': 'req-ac12e3f7-8d42-4a93-a73a-6075152fffc1'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:10.073327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7228d-afaf-4d2d-92f6-6fb6f5bdd603 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:10.078420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbb475-fc76-4b9e-a6a7-acea591bd1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:10.079193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbb475-fc76-4b9e-a6a7-acea591bd1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.079480 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbb475-fc76-4b9e-a6a7-acea591bd1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.079755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbb475-fc76-4b9e-a6a7-acea591bd1b5 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.080565 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbb475-fc76-4b9e-a6a7-acea591bd1b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:10.083364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-ac12e3f7-8d42-4a93-a73a-6075152fffc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a1c131dda542fcabe925a13472228b', 'request_id': 'req-ac12e3f7-8d42-4a93-a73a-6075152fffc1'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:10.085766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbb475-fc76-4b9e-a6a7-acea591bd1b5 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:10.098898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-ac12e3f7-8d42-4a93-a73a-6075152fffc1 admin admin] The token cache is being invalidate because project 42a1c131dda542fcabe925a13472228b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:03:10.099406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-ac12e3f7-8d42-4a93-a73a-6075152fffc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:10.100827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-ac12e3f7-8d42-4a93-a73a-6075152fffc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:10.104087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1247/2528] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:03:09 2022] DELETE /identity/v3/projects/42a1c131dda542fcabe925a13472228b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:10.109825 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cce704-81d8-4b7e-b81c-84a8ccc1ebd1 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.113565 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cce704-81d8-4b7e-b81c-84a8ccc1ebd1 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.113565 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cce704-81d8-4b7e-b81c-84a8ccc1ebd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.135559 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-50cce704-81d8-4b7e-b81c-84a8ccc1ebd1 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:10.136763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-50cce704-81d8-4b7e-b81c-84a8ccc1ebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:10.146559 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1248/2529] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:10.155043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38e51d-6fb1-4707-82a2-0d6ad2156a7a None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.155381 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38e51d-6fb1-4707-82a2-0d6ad2156a7a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.155661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38e51d-6fb1-4707-82a2-0d6ad2156a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.161526 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1282/2530] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:03:09 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:10.165338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbb475-fc76-4b9e-a6a7-acea591bd1b5 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:10.167090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1ebbb475-fc76-4b9e-a6a7-acea591bd1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:10.181371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-fc38e51d-6fb1-4707-82a2-0d6ad2156a7a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:10.181594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbb475-fc76-4b9e-a6a7-acea591bd1b5 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:10.182612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-fc38e51d-6fb1-4707-82a2-0d6ad2156a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:10.187178 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de000159-d41f-4ce4-8f94-651384444111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:10.187781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de000159-d41f-4ce4-8f94-651384444111 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.188056 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de000159-d41f-4ce4-8f94-651384444111 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.188352 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de000159-d41f-4ce4-8f94-651384444111 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.188963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de000159-d41f-4ce4-8f94-651384444111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:10.190706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de000159-d41f-4ce4-8f94-651384444111 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:10.191681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1249/2531] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:10.195364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:10.195780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38e51d-6fb1-4707-82a2-0d6ad2156a7a None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:10.205727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38e51d-6fb1-4707-82a2-0d6ad2156a7a None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:10.207782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1283/2532] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:03:10.210988 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de000159-d41f-4ce4-8f94-651384444111 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:10.211852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d803ac-1e46-4241-8f71-c90e2aadd582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:10.212405 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d803ac-1e46-4241-8f71-c90e2aadd582 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.212675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d803ac-1e46-4241-8f71-c90e2aadd582 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.212947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d803ac-1e46-4241-8f71-c90e2aadd582 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.213635 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d803ac-1e46-4241-8f71-c90e2aadd582 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:10.213929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d803ac-1e46-4241-8f71-c90e2aadd582 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:10.215554 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d803ac-1e46-4241-8f71-c90e2aadd582 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:10.222656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de000159-d41f-4ce4-8f94-651384444111 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:10.227715 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52757e33-d3d4-4a68-a2d8-ad3066e0be56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:10.228358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52757e33-d3d4-4a68-a2d8-ad3066e0be56 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.228641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52757e33-d3d4-4a68-a2d8-ad3066e0be56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.228920 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52757e33-d3d4-4a68-a2d8-ad3066e0be56 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.229526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1250/2533] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:03:10.229652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52757e33-d3d4-4a68-a2d8-ad3066e0be56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:10.231725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52757e33-d3d4-4a68-a2d8-ad3066e0be56 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:10.233931 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d803ac-1e46-4241-8f71-c90e2aadd582 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:10.243471 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1284/2534] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:03:10.248697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d803ac-1e46-4241-8f71-c90e2aadd582 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:10.248886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52757e33-d3d4-4a68-a2d8-ad3066e0be56 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:10.254643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb6e7a-d913-4add-8d6c-48e84340cd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:10.255736 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb6e7a-d913-4add-8d6c-48e84340cd1e admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.256172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb6e7a-d913-4add-8d6c-48e84340cd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.257213 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb6e7a-d913-4add-8d6c-48e84340cd1e admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.258023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb6e7a-d913-4add-8d6c-48e84340cd1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:10.259970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb6e7a-d913-4add-8d6c-48e84340cd1e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:10.261659 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52757e33-d3d4-4a68-a2d8-ad3066e0be56 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:10.267323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219c5f2-bd5d-491f-b64a-2f4667361ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:10.269451 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219c5f2-bd5d-491f-b64a-2f4667361ead admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.269831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219c5f2-bd5d-491f-b64a-2f4667361ead admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.270224 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219c5f2-bd5d-491f-b64a-2f4667361ead admin admin] PATH_INFO: `/v3/projects/18eb044852c6471e83c6b4aa62d6ce96/users/e877ed5efb164c91a5d5cdaaed2f25e6/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.287869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219c5f2-bd5d-491f-b64a-2f4667361ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=18eb044852c6471e83c6b4aa62d6ce96, user_id=e877ed5efb164c91a5d5cdaaed2f25e6, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:10.290560 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219c5f2-bd5d-491f-b64a-2f4667361ead admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:10.373985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1251/2535] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:03:10 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:03:10.375032 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb6e7a-d913-4add-8d6c-48e84340cd1e admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:10.393802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1285/2536] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:10 2022] PUT /identity/v3/projects/18eb044852c6471e83c6b4aa62d6ce96/users/e877ed5efb164c91a5d5cdaaed2f25e6/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:10.406135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffde74-c84d-4938-b7ae-941a3d2d8284 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.406437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffde74-c84d-4938-b7ae-941a3d2d8284 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.406755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffde74-c84d-4938-b7ae-941a3d2d8284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.440373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-71ffde74-c84d-4938-b7ae-941a3d2d8284 None None] MFA Rules not processed for user `e877ed5efb164c91a5d5cdaaed2f25e6`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:10.441604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-71ffde74-c84d-4938-b7ae-941a3d2d8284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:10.456127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb6e7a-d913-4add-8d6c-48e84340cd1e admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:10.470685 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:10.520266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1286/2537] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:10.534585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e64ee2-4959-484c-a02a-4c1806ed1f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:10.535210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e64ee2-4959-484c-a02a-4c1806ed1f8f admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.535496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e64ee2-4959-484c-a02a-4c1806ed1f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.535774 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e64ee2-4959-484c-a02a-4c1806ed1f8f admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.537544 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e64ee2-4959-484c-a02a-4c1806ed1f8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:10.537644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffde74-c84d-4938-b7ae-941a3d2d8284 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:10.538972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-71ffde74-c84d-4938-b7ae-941a3d2d8284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:10.543621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e64ee2-4959-484c-a02a-4c1806ed1f8f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:10.628069 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffde74-c84d-4938-b7ae-941a3d2d8284 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:10.634724 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d09a48-a3a3-4fdb-b6b5-8f792e6488ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:10.635288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d09a48-a3a3-4fdb-b6b5-8f792e6488ac service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.635572 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d09a48-a3a3-4fdb-b6b5-8f792e6488ac service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.635843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d09a48-a3a3-4fdb-b6b5-8f792e6488ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.645382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d09a48-a3a3-4fdb-b6b5-8f792e6488ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:10.647107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d09a48-a3a3-4fdb-b6b5-8f792e6488ac service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:10.663958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1287/2538] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:03:10.671481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1252/2539] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:03:10 2022] POST /identity/v3/users => generated 287 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:10.680394 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d09a48-a3a3-4fdb-b6b5-8f792e6488ac service nova] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:10.694145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d09a48-a3a3-4fdb-b6b5-8f792e6488ac service nova] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:10.699564 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff674d-fd13-4c9a-a618-ce37419b3642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:10.700301 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff674d-fd13-4c9a-a618-ce37419b3642 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.702422 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff674d-fd13-4c9a-a618-ce37419b3642 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.702749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff674d-fd13-4c9a-a618-ce37419b3642 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.703866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff674d-fd13-4c9a-a618-ce37419b3642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:10.706887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff674d-fd13-4c9a-a618-ce37419b3642 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:10.727407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1288/2540] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:10.729178 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e64ee2-4959-484c-a02a-4c1806ed1f8f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:10.743912 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e64ee2-4959-484c-a02a-4c1806ed1f8f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:10.751165 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed679f57-b19f-4420-bc11-7ec2b7cd50a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:10.753942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed679f57-b19f-4420-bc11-7ec2b7cd50a3 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.754225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed679f57-b19f-4420-bc11-7ec2b7cd50a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.754549 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed679f57-b19f-4420-bc11-7ec2b7cd50a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.755188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed679f57-b19f-4420-bc11-7ec2b7cd50a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:10.756795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed679f57-b19f-4420-bc11-7ec2b7cd50a3 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:10.771818 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1253/2541] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:10.773024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff674d-fd13-4c9a-a618-ce37419b3642 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:10.784989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff674d-fd13-4c9a-a618-ce37419b3642 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:10.790187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2cb07-de47-4332-b946-8866c6bb954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:10.791002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2cb07-de47-4332-b946-8866c6bb954d admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.791358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2cb07-de47-4332-b946-8866c6bb954d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.791954 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2cb07-de47-4332-b946-8866c6bb954d admin admin] PATH_INFO: `/v3/projects/bac72e3d134b43f0a339fdcc205565ce/users/d06c36882d504f0a9ba9189239e266c8/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.808891 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2cb07-de47-4332-b946-8866c6bb954d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bac72e3d134b43f0a339fdcc205565ce, user_id=d06c36882d504f0a9ba9189239e266c8, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:10.813066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2cb07-de47-4332-b946-8866c6bb954d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:10.879960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:10.886344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1289/2542] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:10 2022] PUT /identity/v3/projects/bac72e3d134b43f0a339fdcc205565ce/users/d06c36882d504f0a9ba9189239e266c8/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:10.892049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999f7c-4cd5-4a6b-9533-e296c4fb44ac None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:10.892468 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999f7c-4cd5-4a6b-9533-e296c4fb44ac None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:10.892817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999f7c-4cd5-4a6b-9533-e296c4fb44ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:10.915440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-59999f7c-4cd5-4a6b-9533-e296c4fb44ac None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:10.916805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-59999f7c-4cd5-4a6b-9533-e296c4fb44ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:10.995173 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1254/2543] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:11.005423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2cb07-de47-4332-b946-8866c6bb954d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:11.008747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9be2cb07-de47-4332-b946-8866c6bb954d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:11.088809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2cb07-de47-4332-b946-8866c6bb954d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:11.093363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbd4fb-1246-4af1-88e4-91364a61185d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:11.094041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbd4fb-1246-4af1-88e4-91364a61185d service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:11.094449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbd4fb-1246-4af1-88e4-91364a61185d service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:11.094771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbd4fb-1246-4af1-88e4-91364a61185d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:11.105224 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecbd4fb-1246-4af1-88e4-91364a61185d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:11.107516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecbd4fb-1246-4af1-88e4-91364a61185d service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:11.118071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1290/2544] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:12.459662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2246260-9384-458d-a7dc-e268238c4199 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:12.460263 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2246260-9384-458d-a7dc-e268238c4199 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:12.460548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2246260-9384-458d-a7dc-e268238c4199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:12.526542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-f2246260-9384-458d-a7dc-e268238c4199 None None] MFA Rules not processed for user `d06c36882d504f0a9ba9189239e266c8`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:12.529286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f2246260-9384-458d-a7dc-e268238c4199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:12.625568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1255/2545] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:12.630257 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbd4fb-1246-4af1-88e4-91364a61185d service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:12.648092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbd4fb-1246-4af1-88e4-91364a61185d service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:12.648092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d1040-ce52-4146-8186-0552fead0f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:12.648092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d1040-ce52-4146-8186-0552fead0f89 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:12.648092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d1040-ce52-4146-8186-0552fead0f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:12.648411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d1040-ce52-4146-8186-0552fead0f89 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:12.648411 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d1040-ce52-4146-8186-0552fead0f89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:12.649807 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d1040-ce52-4146-8186-0552fead0f89 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:12.791298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1291/2546] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:03:12 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:12.795446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2246260-9384-458d-a7dc-e268238c4199 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:12.806607 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2246260-9384-458d-a7dc-e268238c4199 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:12.828409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:12.874185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ebc703-3f61-45c2-9ed6-60d16b49174e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:12.874914 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebc703-3f61-45c2-9ed6-60d16b49174e admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:12.875200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebc703-3f61-45c2-9ed6-60d16b49174e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:12.875917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebc703-3f61-45c2-9ed6-60d16b49174e admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:12.876299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ebc703-3f61-45c2-9ed6-60d16b49174e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:12.878285 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ebc703-3f61-45c2-9ed6-60d16b49174e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:12.991873 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1256/2547] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:03:12 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:12.992904 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d1040-ce52-4146-8186-0552fead0f89 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:13.004544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d1040-ce52-4146-8186-0552fead0f89 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:13.009366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab80ea-5ea0-4a51-a95e-c5a318095bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:13.010190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab80ea-5ea0-4a51-a95e-c5a318095bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:13.013588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab80ea-5ea0-4a51-a95e-c5a318095bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:13.013588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab80ea-5ea0-4a51-a95e-c5a318095bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:13.013588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab80ea-5ea0-4a51-a95e-c5a318095bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:13.014179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab80ea-5ea0-4a51-a95e-c5a318095bd3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:13.024873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1292/2548] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:13.029854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ebc703-3f61-45c2-9ed6-60d16b49174e admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:13.040217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ebc703-3f61-45c2-9ed6-60d16b49174e admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:13.045088 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d99a5-7cb8-4601-85a0-fcecc081bc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:13.045738 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d99a5-7cb8-4601-85a0-fcecc081bc51 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:13.046015 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d99a5-7cb8-4601-85a0-fcecc081bc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:13.046286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d99a5-7cb8-4601-85a0-fcecc081bc51 admin admin] PATH_INFO: `/v3/projects/bafc51e9d4684d719aa0258361f3f78e/users/52e4f86fe42c4dba9e23581b7eb8b611/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:13.061735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d99a5-7cb8-4601-85a0-fcecc081bc51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bafc51e9d4684d719aa0258361f3f78e, user_id=52e4f86fe42c4dba9e23581b7eb8b611, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:13.064371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d99a5-7cb8-4601-85a0-fcecc081bc51 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:13.134876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1257/2549] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:13 2022] PUT /identity/v3/projects/bafc51e9d4684d719aa0258361f3f78e/users/52e4f86fe42c4dba9e23581b7eb8b611/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:13.138913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab80ea-5ea0-4a51-a95e-c5a318095bd3 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:13.149019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab80ea-5ea0-4a51-a95e-c5a318095bd3 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:13.207767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a645e-f453-465d-8981-9188101b3c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:13.208418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a645e-f453-465d-8981-9188101b3c11 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:13.208703 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a645e-f453-465d-8981-9188101b3c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:13.208975 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a645e-f453-465d-8981-9188101b3c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:13.209616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a645e-f453-465d-8981-9188101b3c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:13.211295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a645e-f453-465d-8981-9188101b3c11 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:13.221819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1293/2550] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:13.226275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d99a5-7cb8-4601-85a0-fcecc081bc51 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:13.242100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d99a5-7cb8-4601-85a0-fcecc081bc51 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:13.247701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1ba8e-2bed-4846-81b6-c4a2b2787274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:13.248370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1ba8e-2bed-4846-81b6-c4a2b2787274 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:13.248653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1ba8e-2bed-4846-81b6-c4a2b2787274 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:13.248930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1ba8e-2bed-4846-81b6-c4a2b2787274 admin admin] PATH_INFO: `/v3/projects/bafc51e9d4684d719aa0258361f3f78e/users/52e4f86fe42c4dba9e23581b7eb8b611/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:13.250913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1ba8e-2bed-4846-81b6-c4a2b2787274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bafc51e9d4684d719aa0258361f3f78e, user_id=52e4f86fe42c4dba9e23581b7eb8b611, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:13.253520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1ba8e-2bed-4846-81b6-c4a2b2787274 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:13.264510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1258/2551] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:13 2022] PUT /identity/v3/projects/bafc51e9d4684d719aa0258361f3f78e/users/52e4f86fe42c4dba9e23581b7eb8b611/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:13.795776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a645e-f453-465d-8981-9188101b3c11 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:13.797612 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9c4a645e-f453-465d-8981-9188101b3c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:13.855775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a645e-f453-465d-8981-9188101b3c11 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:13.874504 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:13.913946 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bf4a7-ab83-4775-b46f-c741fa1b12d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:13.914964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bf4a7-ab83-4775-b46f-c741fa1b12d1 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:13.915773 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bf4a7-ab83-4775-b46f-c741fa1b12d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:13.916047 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bf4a7-ab83-4775-b46f-c741fa1b12d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:13.918534 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-489bf4a7-ab83-4775-b46f-c741fa1b12d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:13.979614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1ba8e-2bed-4846-81b6-c4a2b2787274 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:14.002710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bf4a7-ab83-4775-b46f-c741fa1b12d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:14.004128 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bf4a7-ab83-4775-b46f-c741fa1b12d1 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:14.015763 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1294/2552] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:14.050475 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1ba8e-2bed-4846-81b6-c4a2b2787274 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:14.056407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:14.057132 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:14.057412 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:14.057683 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] PATH_INFO: `/v3/users/e877ed5efb164c91a5d5cdaaed2f25e6` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:14.066546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e877ed5efb164c91a5d5cdaaed2f25e6)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:14.066546 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:14.258588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e877ed5efb164c91a5d5cdaaed2f25e6'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:14.329663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e877ed5efb164c91a5d5cdaaed2f25e6'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:14.330614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e877ed5efb164c91a5d5cdaaed2f25e6'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:14.331998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e877ed5efb164c91a5d5cdaaed2f25e6'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:14.341978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e877ed5efb164c91a5d5cdaaed2f25e6'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:14.358148 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:14.375663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1259/2553] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:03:13 2022] DELETE /identity/v3/users/e877ed5efb164c91a5d5cdaaed2f25e6 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:14.380020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1ae72-453f-49ac-bca9-9ae74cef3fec None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:14.384442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1ae72-453f-49ac-bca9-9ae74cef3fec None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:14.384442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1ae72-453f-49ac-bca9-9ae74cef3fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:14.400578 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-aac1ae72-453f-49ac-bca9-9ae74cef3fec None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:14.401856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-aac1ae72-453f-49ac-bca9-9ae74cef3fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:14.472998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1295/2554] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:14.483177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:14.486978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:14.550884 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19405b1b-6b47-4267-a16e-784f8ffd0fbf admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:14.554660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926c60d-74f3-44dd-80ef-fff56a4eb66a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:14.555266 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926c60d-74f3-44dd-80ef-fff56a4eb66a service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:14.555557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926c60d-74f3-44dd-80ef-fff56a4eb66a service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:14.555826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926c60d-74f3-44dd-80ef-fff56a4eb66a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:14.565781 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926c60d-74f3-44dd-80ef-fff56a4eb66a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:14.567178 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926c60d-74f3-44dd-80ef-fff56a4eb66a service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:14.577178 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1260/2555] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:14.697192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87181b8-6895-413e-adf2-7140a7e38346 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:14.697485 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87181b8-6895-413e-adf2-7140a7e38346 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:14.697759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87181b8-6895-413e-adf2-7140a7e38346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:14.735650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-c87181b8-6895-413e-adf2-7140a7e38346 None None] MFA Rules not processed for user `52e4f86fe42c4dba9e23581b7eb8b611`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:14.737124 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c87181b8-6895-413e-adf2-7140a7e38346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:14.825929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:14.844780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1296/2556] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:14.858771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926c60d-74f3-44dd-80ef-fff56a4eb66a service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:14.859285 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3926c60d-74f3-44dd-80ef-fff56a4eb66a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:14.940725 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926c60d-74f3-44dd-80ef-fff56a4eb66a service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:14.944249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649f83d-0ddd-45cc-9402-f903d5059b4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:14.945028 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649f83d-0ddd-45cc-9402-f903d5059b4e service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:14.945361 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649f83d-0ddd-45cc-9402-f903d5059b4e service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:14.945776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649f83d-0ddd-45cc-9402-f903d5059b4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:14.947736 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2649f83d-0ddd-45cc-9402-f903d5059b4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:15.017713 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649f83d-0ddd-45cc-9402-f903d5059b4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:15.019191 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649f83d-0ddd-45cc-9402-f903d5059b4e service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:15.029493 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1261/2557] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:15.046375 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87181b8-6895-413e-adf2-7140a7e38346 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:15.048465 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c87181b8-6895-413e-adf2-7140a7e38346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:15.088101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87181b8-6895-413e-adf2-7140a7e38346 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:15.096112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680408c-1afa-4ec4-bed3-59d44eda3d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:15.096917 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680408c-1afa-4ec4-bed3-59d44eda3d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:15.097330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680408c-1afa-4ec4-bed3-59d44eda3d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:15.097733 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680408c-1afa-4ec4-bed3-59d44eda3d6a admin admin] PATH_INFO: `/v3/projects/18eb044852c6471e83c6b4aa62d6ce96` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:15.099379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680408c-1afa-4ec4-bed3-59d44eda3d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=18eb044852c6471e83c6b4aa62d6ce96)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:15.101550 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680408c-1afa-4ec4-bed3-59d44eda3d6a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:15.195843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-e680408c-1afa-4ec4-bed3-59d44eda3d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18eb044852c6471e83c6b4aa62d6ce96', 'request_id': 'req-e680408c-1afa-4ec4-bed3-59d44eda3d6a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:15.202967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-e680408c-1afa-4ec4-bed3-59d44eda3d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18eb044852c6471e83c6b4aa62d6ce96', 'request_id': 'req-e680408c-1afa-4ec4-bed3-59d44eda3d6a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:15.260811 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-e680408c-1afa-4ec4-bed3-59d44eda3d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18eb044852c6471e83c6b4aa62d6ce96', 'request_id': 'req-e680408c-1afa-4ec4-bed3-59d44eda3d6a'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:15.277141 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-e680408c-1afa-4ec4-bed3-59d44eda3d6a admin admin] The token cache is being invalidate because project 18eb044852c6471e83c6b4aa62d6ce96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:03:15.277827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-e680408c-1afa-4ec4-bed3-59d44eda3d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:15.278572 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-e680408c-1afa-4ec4-bed3-59d44eda3d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:15.280237 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1297/2558] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:03:15 2022] DELETE /identity/v3/projects/18eb044852c6471e83c6b4aa62d6ce96 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:15.284998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cef5b6-2afd-4dfc-a4f7-7c92551d4ba4 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:15.285364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cef5b6-2afd-4dfc-a4f7-7c92551d4ba4 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:15.285715 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cef5b6-2afd-4dfc-a4f7-7c92551d4ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:15.306176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-23cef5b6-2afd-4dfc-a4f7-7c92551d4ba4 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:15.307409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-23cef5b6-2afd-4dfc-a4f7-7c92551d4ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:15.357950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:15.371123 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1262/2559] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:15.380913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6ecf4-3908-4f73-89d4-0a98ff1207fb None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:15.381216 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6ecf4-3908-4f73-89d4-0a98ff1207fb None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:15.381496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6ecf4-3908-4f73-89d4-0a98ff1207fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:15.402316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-15d6ecf4-3908-4f73-89d4-0a98ff1207fb None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:15.403586 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-15d6ecf4-3908-4f73-89d4-0a98ff1207fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:15.411775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1298/2560] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:15.415880 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cef5b6-2afd-4dfc-a4f7-7c92551d4ba4 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:15.426012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cef5b6-2afd-4dfc-a4f7-7c92551d4ba4 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:15.430672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cfea2-0299-4684-956b-f3f43517d5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:15.431272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cfea2-0299-4684-956b-f3f43517d5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:15.431555 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cfea2-0299-4684-956b-f3f43517d5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:15.431830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cfea2-0299-4684-956b-f3f43517d5f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:15.432471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cfea2-0299-4684-956b-f3f43517d5f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:15.432802 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cfea2-0299-4684-956b-f3f43517d5f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:15.434979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cfea2-0299-4684-956b-f3f43517d5f4 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:15.452520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1263/2561] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:15.453761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6ecf4-3908-4f73-89d4-0a98ff1207fb None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:15.463379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6ecf4-3908-4f73-89d4-0a98ff1207fb None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:15.468584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a213457-024e-4a1e-9aa9-2076fbcebe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:15.469231 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a213457-024e-4a1e-9aa9-2076fbcebe11 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:15.469513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a213457-024e-4a1e-9aa9-2076fbcebe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:15.469785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a213457-024e-4a1e-9aa9-2076fbcebe11 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:15.470416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a213457-024e-4a1e-9aa9-2076fbcebe11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:15.472432 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a213457-024e-4a1e-9aa9-2076fbcebe11 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:15.600449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1299/2562] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:03:15 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:15.602929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cfea2-0299-4684-956b-f3f43517d5f4 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:15.607068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a213457-024e-4a1e-9aa9-2076fbcebe11 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:15.608904 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1a213457-024e-4a1e-9aa9-2076fbcebe11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:15.610932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cfea2-0299-4684-956b-f3f43517d5f4 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:15.668406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a213457-024e-4a1e-9aa9-2076fbcebe11 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:15.671812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87c239-c009-478c-9d3b-a184801d1f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:15.672314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865ae1d-6513-44bd-8aaf-89e2ffec455f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:15.672475 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87c239-c009-478c-9d3b-a184801d1f6a admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:15.672691 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87c239-c009-478c-9d3b-a184801d1f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:15.672915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865ae1d-6513-44bd-8aaf-89e2ffec455f service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:15.673020 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87c239-c009-478c-9d3b-a184801d1f6a admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:15.673191 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865ae1d-6513-44bd-8aaf-89e2ffec455f service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:15.673524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865ae1d-6513-44bd-8aaf-89e2ffec455f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:15.673630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87c239-c009-478c-9d3b-a184801d1f6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:15.675330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1865ae1d-6513-44bd-8aaf-89e2ffec455f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:15.675686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87c239-c009-478c-9d3b-a184801d1f6a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:15.706280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865ae1d-6513-44bd-8aaf-89e2ffec455f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:15.709942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865ae1d-6513-44bd-8aaf-89e2ffec455f service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:15.719508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1300/2563] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:03:15.770895 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1264/2564] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:03:15 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:15.772999 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865ae1d-6513-44bd-8aaf-89e2ffec455f service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:15.784274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865ae1d-6513-44bd-8aaf-89e2ffec455f service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:15.810203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360f0f6-b76c-47d1-9f74-199e92128845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:15.811025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360f0f6-b76c-47d1-9f74-199e92128845 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:15.811409 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360f0f6-b76c-47d1-9f74-199e92128845 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:15.811784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360f0f6-b76c-47d1-9f74-199e92128845 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:15.812551 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360f0f6-b76c-47d1-9f74-199e92128845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:15.814416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360f0f6-b76c-47d1-9f74-199e92128845 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:15.837242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87c239-c009-478c-9d3b-a184801d1f6a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:15.846968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1301/2565] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:15.852928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87c239-c009-478c-9d3b-a184801d1f6a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:15.861656 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1bf4f-ca5c-46f8-9ed7-d672568dd62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:15.862562 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1bf4f-ca5c-46f8-9ed7-d672568dd62e admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:15.862851 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1bf4f-ca5c-46f8-9ed7-d672568dd62e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:15.863179 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1bf4f-ca5c-46f8-9ed7-d672568dd62e admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:15.864026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1bf4f-ca5c-46f8-9ed7-d672568dd62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:15.866630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1bf4f-ca5c-46f8-9ed7-d672568dd62e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:15.882599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1265/2566] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:15.887925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360f0f6-b76c-47d1-9f74-199e92128845 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:15.901087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360f0f6-b76c-47d1-9f74-199e92128845 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:15.907376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae996ae8-a703-4476-89c8-c15c76ec5184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:15.908221 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae996ae8-a703-4476-89c8-c15c76ec5184 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:15.908887 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae996ae8-a703-4476-89c8-c15c76ec5184 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:15.909716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae996ae8-a703-4476-89c8-c15c76ec5184 admin admin] PATH_INFO: `/v3/projects/68f25a496fe247088fbe40c155b109c6/users/42cfa3c19bfc4497865c9a1d91cfa550/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:15.928277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae996ae8-a703-4476-89c8-c15c76ec5184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f25a496fe247088fbe40c155b109c6, user_id=42cfa3c19bfc4497865c9a1d91cfa550, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:15.931371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae996ae8-a703-4476-89c8-c15c76ec5184 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:16.038312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1302/2567] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:15 2022] PUT /identity/v3/projects/68f25a496fe247088fbe40c155b109c6/users/42cfa3c19bfc4497865c9a1d91cfa550/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:16.050754 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66627f13-6ce4-4d52-b05f-a771006ef5cd None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:16.052141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66627f13-6ce4-4d52-b05f-a771006ef5cd None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:16.052613 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66627f13-6ce4-4d52-b05f-a771006ef5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:16.084177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-66627f13-6ce4-4d52-b05f-a771006ef5cd None None] MFA Rules not processed for user `42cfa3c19bfc4497865c9a1d91cfa550`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:16.085483 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-66627f13-6ce4-4d52-b05f-a771006ef5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:16.169719 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1266/2568] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:16.174515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae996ae8-a703-4476-89c8-c15c76ec5184 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:16.198172 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:16.248334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae996ae8-a703-4476-89c8-c15c76ec5184 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:16.253646 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ed55d-1efa-450d-9788-b9286b50b9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:16.256206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ed55d-1efa-450d-9788-b9286b50b9bb admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:16.256206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ed55d-1efa-450d-9788-b9286b50b9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:16.256206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ed55d-1efa-450d-9788-b9286b50b9bb admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:16.256206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ed55d-1efa-450d-9788-b9286b50b9bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:16.258370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ed55d-1efa-450d-9788-b9286b50b9bb admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:16.367785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66627f13-6ce4-4d52-b05f-a771006ef5cd None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:16.368195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1303/2569] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:03:16 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:16.387880 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66627f13-6ce4-4d52-b05f-a771006ef5cd None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:16.462780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:16.476962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba646578-950c-4f20-9079-c8544c278440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:16.477831 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba646578-950c-4f20-9079-c8544c278440 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:16.478330 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba646578-950c-4f20-9079-c8544c278440 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:16.478713 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba646578-950c-4f20-9079-c8544c278440 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:16.479540 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba646578-950c-4f20-9079-c8544c278440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:16.482145 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba646578-950c-4f20-9079-c8544c278440 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:16.583696 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1267/2570] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:03:16 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:03:16.587495 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ed55d-1efa-450d-9788-b9286b50b9bb admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:16.608950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ed55d-1efa-450d-9788-b9286b50b9bb admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:16.616622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f66064-3114-4ec4-ba69-e75273120faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:16.617777 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f66064-3114-4ec4-ba69-e75273120faa admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:16.617777 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f66064-3114-4ec4-ba69-e75273120faa admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:16.618052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f66064-3114-4ec4-ba69-e75273120faa admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:16.619290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f66064-3114-4ec4-ba69-e75273120faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:16.621379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f66064-3114-4ec4-ba69-e75273120faa admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:16.640295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1304/2571] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:16.640983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba646578-950c-4f20-9079-c8544c278440 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:16.656936 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba646578-950c-4f20-9079-c8544c278440 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:16.664303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f8e85-4e64-45e1-9929-d676581516dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:16.666571 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f8e85-4e64-45e1-9929-d676581516dd admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:16.666571 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f8e85-4e64-45e1-9929-d676581516dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:16.666571 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f8e85-4e64-45e1-9929-d676581516dd admin admin] PATH_INFO: `/v3/projects/2faf81e109644e06a89dd7160e6fb7c6/users/6948ebc008c9483d9aac480644f4ff4b/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:16.686446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f8e85-4e64-45e1-9929-d676581516dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2faf81e109644e06a89dd7160e6fb7c6, user_id=6948ebc008c9483d9aac480644f4ff4b, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:16.690769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f8e85-4e64-45e1-9929-d676581516dd admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:16.774302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1268/2572] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:16 2022] PUT /identity/v3/projects/2faf81e109644e06a89dd7160e6fb7c6/users/6948ebc008c9483d9aac480644f4ff4b/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:03:16.777792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f66064-3114-4ec4-ba69-e75273120faa admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:16.792032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f66064-3114-4ec4-ba69-e75273120faa admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:16.830087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f8e85-4e64-45e1-9929-d676581516dd admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:16.833811 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b00f8e85-4e64-45e1-9929-d676581516dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:16.894360 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265292b-7bf9-45ae-9a15-5a89046358da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:16.895723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265292b-7bf9-45ae-9a15-5a89046358da admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:16.896104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265292b-7bf9-45ae-9a15-5a89046358da admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:16.897405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265292b-7bf9-45ae-9a15-5a89046358da admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:16.898594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265292b-7bf9-45ae-9a15-5a89046358da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:16.901130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265292b-7bf9-45ae-9a15-5a89046358da admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:16.917667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1305/2573] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:03:16.922067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265292b-7bf9-45ae-9a15-5a89046358da admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:16.924796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f8e85-4e64-45e1-9929-d676581516dd admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:16.931127 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6188e52-4656-4010-ade4-d7db7a21b6bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:16.931926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6188e52-4656-4010-ade4-d7db7a21b6bc service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:16.932217 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6188e52-4656-4010-ade4-d7db7a21b6bc service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:16.932498 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6188e52-4656-4010-ade4-d7db7a21b6bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:16.933144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265292b-7bf9-45ae-9a15-5a89046358da admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:16.935160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e6188e52-4656-4010-ade4-d7db7a21b6bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:16.938809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a590a0-c306-4dba-a01f-f6f664df5540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:16.939710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a590a0-c306-4dba-a01f-f6f664df5540 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:16.939996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a590a0-c306-4dba-a01f-f6f664df5540 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:16.940340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a590a0-c306-4dba-a01f-f6f664df5540 admin admin] PATH_INFO: `/v3/projects/2faf81e109644e06a89dd7160e6fb7c6/users/6948ebc008c9483d9aac480644f4ff4b/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:16.942305 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a590a0-c306-4dba-a01f-f6f664df5540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2faf81e109644e06a89dd7160e6fb7c6, user_id=6948ebc008c9483d9aac480644f4ff4b, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:16.945872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a590a0-c306-4dba-a01f-f6f664df5540 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:16.961763 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1306/2574] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:16 2022] PUT /identity/v3/projects/2faf81e109644e06a89dd7160e6fb7c6/users/6948ebc008c9483d9aac480644f4ff4b/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:16.972656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b824ec-c655-4a7e-ac43-90c247274e8b None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:16.973013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b824ec-c655-4a7e-ac43-90c247274e8b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:16.973488 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b824ec-c655-4a7e-ac43-90c247274e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:17.007792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-45b824ec-c655-4a7e-ac43-90c247274e8b None None] MFA Rules not processed for user `6948ebc008c9483d9aac480644f4ff4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:17.009241 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-45b824ec-c655-4a7e-ac43-90c247274e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:17.093552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6188e52-4656-4010-ade4-d7db7a21b6bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:17.096012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6188e52-4656-4010-ade4-d7db7a21b6bc service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:17.118211 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1269/2575] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:03:17.149476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1307/2576] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:17.158818 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6188e52-4656-4010-ade4-d7db7a21b6bc service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:17.161202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e6188e52-4656-4010-ade4-d7db7a21b6bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:17.174442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6188e52-4656-4010-ade4-d7db7a21b6bc service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:17.265285 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3698de2-4d05-4bd7-82f5-59a402c9d12b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:17.266203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3698de2-4d05-4bd7-82f5-59a402c9d12b service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:17.271533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3698de2-4d05-4bd7-82f5-59a402c9d12b service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:17.271533 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3698de2-4d05-4bd7-82f5-59a402c9d12b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:17.314990 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:17.408044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3698de2-4d05-4bd7-82f5-59a402c9d12b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:17.408044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3698de2-4d05-4bd7-82f5-59a402c9d12b service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:17.414564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1270/2577] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:19.961795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b824ec-c655-4a7e-ac43-90c247274e8b None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:19.962984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-45b824ec-c655-4a7e-ac43-90c247274e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:19.969583 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:20.031900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b824ec-c655-4a7e-ac43-90c247274e8b None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:20.036838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e648d-d87b-4da4-8bc8-f6f907e3ae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:20.042206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e648d-d87b-4da4-8bc8-f6f907e3ae44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:20.042206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e648d-d87b-4da4-8bc8-f6f907e3ae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:20.042206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e648d-d87b-4da4-8bc8-f6f907e3ae44 admin admin] PATH_INFO: `/v3/users/6ec951604cde4768beb02906caf6d7c2` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:20.042206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e648d-d87b-4da4-8bc8-f6f907e3ae44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec951604cde4768beb02906caf6d7c2)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:20.042206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e648d-d87b-4da4-8bc8-f6f907e3ae44 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:20.266995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a74e648d-d87b-4da4-8bc8-f6f907e3ae44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec951604cde4768beb02906caf6d7c2'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:20.341227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a74e648d-d87b-4da4-8bc8-f6f907e3ae44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec951604cde4768beb02906caf6d7c2'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:20.341770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a74e648d-d87b-4da4-8bc8-f6f907e3ae44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec951604cde4768beb02906caf6d7c2'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:20.342392 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a74e648d-d87b-4da4-8bc8-f6f907e3ae44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec951604cde4768beb02906caf6d7c2'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:20.355267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a74e648d-d87b-4da4-8bc8-f6f907e3ae44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec951604cde4768beb02906caf6d7c2'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:20.367614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1308/2578] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:03:19 2022] DELETE /identity/v3/users/6ec951604cde4768beb02906caf6d7c2 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:20.616204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3698de2-4d05-4bd7-82f5-59a402c9d12b service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:20.635320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3698de2-4d05-4bd7-82f5-59a402c9d12b service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:20.704438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc812b-0cb0-4716-8990-62c558c1b6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:20.705193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc812b-0cb0-4716-8990-62c558c1b6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:20.705480 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc812b-0cb0-4716-8990-62c558c1b6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:20.705757 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc812b-0cb0-4716-8990-62c558c1b6b2 admin admin] PATH_INFO: `/v3/projects/8880382269b94daa8a25f92178a478d5` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:20.707204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc812b-0cb0-4716-8990-62c558c1b6b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8880382269b94daa8a25f92178a478d5)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:20.709223 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc812b-0cb0-4716-8990-62c558c1b6b2 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:20.828188 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4fbc812b-0cb0-4716-8990-62c558c1b6b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8880382269b94daa8a25f92178a478d5', 'request_id': 'req-4fbc812b-0cb0-4716-8990-62c558c1b6b2'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:20.834507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4fbc812b-0cb0-4716-8990-62c558c1b6b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8880382269b94daa8a25f92178a478d5', 'request_id': 'req-4fbc812b-0cb0-4716-8990-62c558c1b6b2'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:20.900109 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4fbc812b-0cb0-4716-8990-62c558c1b6b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8880382269b94daa8a25f92178a478d5', 'request_id': 'req-4fbc812b-0cb0-4716-8990-62c558c1b6b2'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:20.911463 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4fbc812b-0cb0-4716-8990-62c558c1b6b2 admin admin] The token cache is being invalidate because project 8880382269b94daa8a25f92178a478d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:03:20.911751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4fbc812b-0cb0-4716-8990-62c558c1b6b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:20.912388 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4fbc812b-0cb0-4716-8990-62c558c1b6b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:20.913870 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1271/2579] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:03:20 2022] DELETE /identity/v3/projects/8880382269b94daa8a25f92178a478d5 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:20.919434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb44b8-c09b-44dc-ab08-50b79953c07b None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:20.919716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb44b8-c09b-44dc-ab08-50b79953c07b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:20.920187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb44b8-c09b-44dc-ab08-50b79953c07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:20.944149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-17bb44b8-c09b-44dc-ab08-50b79953c07b None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:20.944149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-17bb44b8-c09b-44dc-ab08-50b79953c07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:21.008182 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1309/2580] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:21.020884 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e7837-ed3c-4e54-a044-30a240f98755 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:21.021171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e7837-ed3c-4e54-a044-30a240f98755 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:21.021444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e7837-ed3c-4e54-a044-30a240f98755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:21.041276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ab3e7837-ed3c-4e54-a044-30a240f98755 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:21.042507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ab3e7837-ed3c-4e54-a044-30a240f98755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:21.050977 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1272/2581] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:21.055291 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb44b8-c09b-44dc-ab08-50b79953c07b None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:21.065812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb44b8-c09b-44dc-ab08-50b79953c07b None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:21.070367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd22f4-8832-4967-a336-4330aaa921bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:21.070992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd22f4-8832-4967-a336-4330aaa921bd admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:21.071270 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd22f4-8832-4967-a336-4330aaa921bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:21.071543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd22f4-8832-4967-a336-4330aaa921bd admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:21.072187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd22f4-8832-4967-a336-4330aaa921bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:21.072490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd22f4-8832-4967-a336-4330aaa921bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:21.074100 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd22f4-8832-4967-a336-4330aaa921bd admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:21.085833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1310/2582] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:21.090049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e7837-ed3c-4e54-a044-30a240f98755 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:21.099333 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e7837-ed3c-4e54-a044-30a240f98755 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:21.103973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b82bc8-b5dc-4fba-90a5-e3bccec4d41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:21.104570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b82bc8-b5dc-4fba-90a5-e3bccec4d41f admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:21.104853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b82bc8-b5dc-4fba-90a5-e3bccec4d41f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:21.105130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b82bc8-b5dc-4fba-90a5-e3bccec4d41f admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:21.105754 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b82bc8-b5dc-4fba-90a5-e3bccec4d41f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:21.108404 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b82bc8-b5dc-4fba-90a5-e3bccec4d41f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:21.193638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1273/2583] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:03:21 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:21.197540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd22f4-8832-4967-a336-4330aaa921bd admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:21.208032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd22f4-8832-4967-a336-4330aaa921bd admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:21.212610 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:21.267736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b0c14-317e-4e63-b41a-fe1add3f9ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:21.268410 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b0c14-317e-4e63-b41a-fe1add3f9ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:21.268690 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b0c14-317e-4e63-b41a-fe1add3f9ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:21.268962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b0c14-317e-4e63-b41a-fe1add3f9ef0 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:21.269608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b0c14-317e-4e63-b41a-fe1add3f9ef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:21.271642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b0c14-317e-4e63-b41a-fe1add3f9ef0 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:21.375175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1311/2584] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:03:21 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:03:21.379460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b82bc8-b5dc-4fba-90a5-e3bccec4d41f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:21.389972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b82bc8-b5dc-4fba-90a5-e3bccec4d41f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:21.394784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f4ab4-482e-42cc-85d7-a042ce67c6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:21.398031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f4ab4-482e-42cc-85d7-a042ce67c6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:21.398031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f4ab4-482e-42cc-85d7-a042ce67c6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:21.398031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f4ab4-482e-42cc-85d7-a042ce67c6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:21.398031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f4ab4-482e-42cc-85d7-a042ce67c6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:21.398389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f4ab4-482e-42cc-85d7-a042ce67c6d9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:21.408685 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1274/2585] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:21.412720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b0c14-317e-4e63-b41a-fe1add3f9ef0 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:21.423836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b0c14-317e-4e63-b41a-fe1add3f9ef0 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:21.428564 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06a0e6-217a-4b2c-b2b5-5db69aa85a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:21.429219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06a0e6-217a-4b2c-b2b5-5db69aa85a97 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:21.429498 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06a0e6-217a-4b2c-b2b5-5db69aa85a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:21.429796 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06a0e6-217a-4b2c-b2b5-5db69aa85a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:21.430434 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06a0e6-217a-4b2c-b2b5-5db69aa85a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:21.432137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06a0e6-217a-4b2c-b2b5-5db69aa85a97 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:21.443632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1312/2586] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:21.447700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f4ab4-482e-42cc-85d7-a042ce67c6d9 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:21.457699 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f4ab4-482e-42cc-85d7-a042ce67c6d9 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:21.462302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7375912-4050-4824-b19e-64bbdbf6aa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:21.462970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7375912-4050-4824-b19e-64bbdbf6aa72 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:21.463252 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7375912-4050-4824-b19e-64bbdbf6aa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:21.463525 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7375912-4050-4824-b19e-64bbdbf6aa72 admin admin] PATH_INFO: `/v3/projects/31f8b13a6a4844fc9fd20ab2c3ff7368/users/bc2cc611fd9341fc9a6d8a5d49f43e53/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:21.467721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:21.478956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7375912-4050-4824-b19e-64bbdbf6aa72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31f8b13a6a4844fc9fd20ab2c3ff7368, user_id=bc2cc611fd9341fc9a6d8a5d49f43e53, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:21.482000 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7375912-4050-4824-b19e-64bbdbf6aa72 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:21.522907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1275/2587] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:21 2022] PUT /identity/v3/projects/31f8b13a6a4844fc9fd20ab2c3ff7368/users/bc2cc611fd9341fc9a6d8a5d49f43e53/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:21.528086 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4292212-763d-4856-a48e-d5f415e753e7 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:21.528271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4292212-763d-4856-a48e-d5f415e753e7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:21.528487 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4292212-763d-4856-a48e-d5f415e753e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:21.548284 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-e4292212-763d-4856-a48e-d5f415e753e7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:21.549483 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e4292212-763d-4856-a48e-d5f415e753e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:21.609052 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1313/2588] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:21.618791 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7375912-4050-4824-b19e-64bbdbf6aa72 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:21.620563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d7375912-4050-4824-b19e-64bbdbf6aa72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:21.677095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7375912-4050-4824-b19e-64bbdbf6aa72 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:21.680778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb86b3-c0e4-4b21-8ded-f1e2dcfa7bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:21.681382 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb86b3-c0e4-4b21-8ded-f1e2dcfa7bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:21.681666 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb86b3-c0e4-4b21-8ded-f1e2dcfa7bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:21.681942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb86b3-c0e4-4b21-8ded-f1e2dcfa7bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:21.691730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb86b3-c0e4-4b21-8ded-f1e2dcfa7bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:21.693167 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb86b3-c0e4-4b21-8ded-f1e2dcfa7bf5 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:21.703329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1276/2589] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:22.998791 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4292212-763d-4856-a48e-d5f415e753e7 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:23.000456 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e4292212-763d-4856-a48e-d5f415e753e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:23.076574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4292212-763d-4856-a48e-d5f415e753e7 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:23.091788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad02937-46bc-405d-8901-e330a32fdc25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:23.092710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad02937-46bc-405d-8901-e330a32fdc25 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:23.092991 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad02937-46bc-405d-8901-e330a32fdc25 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:23.093395 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad02937-46bc-405d-8901-e330a32fdc25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:23.095165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8ad02937-46bc-405d-8901-e330a32fdc25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:23.161662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:23.183022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad02937-46bc-405d-8901-e330a32fdc25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:23.184516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad02937-46bc-405d-8901-e330a32fdc25 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:23.194403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1314/2590] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:27.181534 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1e43-68ea-4093-9822-dd11a021f284 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:27.182102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1e43-68ea-4093-9822-dd11a021f284 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:27.182102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1e43-68ea-4093-9822-dd11a021f284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:27.216729 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-e01f1e43-68ea-4093-9822-dd11a021f284 None None] MFA Rules not processed for user `bc2cc611fd9341fc9a6d8a5d49f43e53`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:27.218108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e01f1e43-68ea-4093-9822-dd11a021f284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:27.303121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1277/2591] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:27.318116 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad02937-46bc-405d-8901-e330a32fdc25 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:27.319619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8ad02937-46bc-405d-8901-e330a32fdc25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:27.413329 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad02937-46bc-405d-8901-e330a32fdc25 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:27.419072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05c488-fe92-491a-9215-e2574c17352b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:27.419694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05c488-fe92-491a-9215-e2574c17352b service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:27.419967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05c488-fe92-491a-9215-e2574c17352b service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:27.420235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05c488-fe92-491a-9215-e2574c17352b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:27.429001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05c488-fe92-491a-9215-e2574c17352b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:27.431529 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05c488-fe92-491a-9215-e2574c17352b service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:27.446657 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1315/2592] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:29.883528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a71f3-992c-4ebf-b58a-e22f033fda49 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:29.884099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a71f3-992c-4ebf-b58a-e22f033fda49 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:29.884099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a71f3-992c-4ebf-b58a-e22f033fda49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:29.915959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-f34a71f3-992c-4ebf-b58a-e22f033fda49 None None] MFA Rules not processed for user `bc2cc611fd9341fc9a6d8a5d49f43e53`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:29.924914 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f34a71f3-992c-4ebf-b58a-e22f033fda49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:29.928376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1278/2593] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:03:30.367652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05c488-fe92-491a-9215-e2574c17352b service nova] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:30.369253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5f05c488-fe92-491a-9215-e2574c17352b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:30.381559 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05c488-fe92-491a-9215-e2574c17352b service nova] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:30.385113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e8c05-bdeb-4d4c-9920-857371a24d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:30.385709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e8c05-bdeb-4d4c-9920-857371a24d17 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:30.385983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e8c05-bdeb-4d4c-9920-857371a24d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:30.386252 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e8c05-bdeb-4d4c-9920-857371a24d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:30.388135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-007e8c05-bdeb-4d4c-9920-857371a24d17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:30.399006 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e8c05-bdeb-4d4c-9920-857371a24d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:30.400514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e8c05-bdeb-4d4c-9920-857371a24d17 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:30.410091 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1316/2594] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:03:39.904978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=67,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:40.406089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=50 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:43.220023 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a71f3-992c-4ebf-b58a-e22f033fda49 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:43.221494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f34a71f3-992c-4ebf-b58a-e22f033fda49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:43.236134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a71f3-992c-4ebf-b58a-e22f033fda49 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:43.241182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:43.245488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:43.245488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:43.245488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] PATH_INFO: `/v3/users/d06c36882d504f0a9ba9189239e266c8` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:43.245488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d06c36882d504f0a9ba9189239e266c8)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:43.245488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:43.446639 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd06c36882d504f0a9ba9189239e266c8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:43.528432 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd06c36882d504f0a9ba9189239e266c8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:43.529064 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd06c36882d504f0a9ba9189239e266c8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:43.529675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd06c36882d504f0a9ba9189239e266c8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:43.537267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd06c36882d504f0a9ba9189239e266c8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:43.557979 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1279/2595] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:03:43 2022] DELETE /identity/v3/users/d06c36882d504f0a9ba9189239e266c8 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:43.793029 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e8c05-bdeb-4d4c-9920-857371a24d17 service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:43.807118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e8c05-bdeb-4d4c-9920-857371a24d17 service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:43.863831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:43.864451 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:43.864830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:43.864996 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] PATH_INFO: `/v3/projects/bac72e3d134b43f0a339fdcc205565ce` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:43.866115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bac72e3d134b43f0a339fdcc205565ce)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:43.868012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:43.964284 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bac72e3d134b43f0a339fdcc205565ce', 'request_id': 'req-945da08d-9363-4b4d-acd1-22a5b3f5806e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:43.970676 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bac72e3d134b43f0a339fdcc205565ce', 'request_id': 'req-945da08d-9363-4b4d-acd1-22a5b3f5806e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:44.059710 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bac72e3d134b43f0a339fdcc205565ce', 'request_id': 'req-945da08d-9363-4b4d-acd1-22a5b3f5806e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:44.071068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] The token cache is being invalidate because project bac72e3d134b43f0a339fdcc205565ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:03:44.071350 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:44.071968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:44.073459 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1317/2596] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:03:43 2022] DELETE /identity/v3/projects/bac72e3d134b43f0a339fdcc205565ce => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:44.077467 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:44.078872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:44.091853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f155982-a783-4743-8f7d-1033cf455fc3 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:44.147899 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98252774-371b-4fcf-95cd-9b084cf04ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:44.148521 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98252774-371b-4fcf-95cd-9b084cf04ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:44.148796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98252774-371b-4fcf-95cd-9b084cf04ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:44.149064 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98252774-371b-4fcf-95cd-9b084cf04ea9 admin admin] PATH_INFO: `/v3/users/52e4f86fe42c4dba9e23581b7eb8b611` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:44.150160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98252774-371b-4fcf-95cd-9b084cf04ea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52e4f86fe42c4dba9e23581b7eb8b611)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:44.152095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98252774-371b-4fcf-95cd-9b084cf04ea9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:44.361429 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-98252774-371b-4fcf-95cd-9b084cf04ea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52e4f86fe42c4dba9e23581b7eb8b611'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:44.448358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-98252774-371b-4fcf-95cd-9b084cf04ea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52e4f86fe42c4dba9e23581b7eb8b611'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:44.449100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-98252774-371b-4fcf-95cd-9b084cf04ea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52e4f86fe42c4dba9e23581b7eb8b611'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:44.449773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-98252774-371b-4fcf-95cd-9b084cf04ea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52e4f86fe42c4dba9e23581b7eb8b611'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:44.457771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-98252774-371b-4fcf-95cd-9b084cf04ea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52e4f86fe42c4dba9e23581b7eb8b611'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:44.466052 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:44.476653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1280/2597] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:03:44 2022] DELETE /identity/v3/users/52e4f86fe42c4dba9e23581b7eb8b611 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:44.719095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:44.724002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:44.734013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945da08d-9363-4b4d-acd1-22a5b3f5806e admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:44.795407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdecf88-6ef3-4a51-886c-3b63a50e976f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:44.796123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdecf88-6ef3-4a51-886c-3b63a50e976f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:44.796300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdecf88-6ef3-4a51-886c-3b63a50e976f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:44.796574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdecf88-6ef3-4a51-886c-3b63a50e976f admin admin] PATH_INFO: `/v3/projects/bafc51e9d4684d719aa0258361f3f78e` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:44.797680 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdecf88-6ef3-4a51-886c-3b63a50e976f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bafc51e9d4684d719aa0258361f3f78e)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:44.799689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdecf88-6ef3-4a51-886c-3b63a50e976f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:44.905985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:44.919171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4fdecf88-6ef3-4a51-886c-3b63a50e976f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bafc51e9d4684d719aa0258361f3f78e', 'request_id': 'req-4fdecf88-6ef3-4a51-886c-3b63a50e976f'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:44.925984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4fdecf88-6ef3-4a51-886c-3b63a50e976f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bafc51e9d4684d719aa0258361f3f78e', 'request_id': 'req-4fdecf88-6ef3-4a51-886c-3b63a50e976f'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:44.991365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4fdecf88-6ef3-4a51-886c-3b63a50e976f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bafc51e9d4684d719aa0258361f3f78e', 'request_id': 'req-4fdecf88-6ef3-4a51-886c-3b63a50e976f'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:45.002046 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4fdecf88-6ef3-4a51-886c-3b63a50e976f admin admin] The token cache is being invalidate because project bafc51e9d4684d719aa0258361f3f78e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:03:45.002337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4fdecf88-6ef3-4a51-886c-3b63a50e976f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:45.003003 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4fdecf88-6ef3-4a51-886c-3b63a50e976f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:45.005132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1318/2598] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:03:44 2022] DELETE /identity/v3/projects/bafc51e9d4684d719aa0258361f3f78e => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:45.010014 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147719ff-c59e-44ce-b591-0a95d7aca3cb None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:45.010304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147719ff-c59e-44ce-b591-0a95d7aca3cb None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:45.010623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147719ff-c59e-44ce-b591-0a95d7aca3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:45.030561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-147719ff-c59e-44ce-b591-0a95d7aca3cb None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:45.031762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-147719ff-c59e-44ce-b591-0a95d7aca3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:45.091947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1281/2599] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:45.100010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6c74a-1d0c-4fa0-a28a-c6fbafaa7340 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:45.100297 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6c74a-1d0c-4fa0-a28a-c6fbafaa7340 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:45.100572 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6c74a-1d0c-4fa0-a28a-c6fbafaa7340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:45.120524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0ce6c74a-1d0c-4fa0-a28a-c6fbafaa7340 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:45.121726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0ce6c74a-1d0c-4fa0-a28a-c6fbafaa7340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:45.130543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1319/2600] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:45.134846 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147719ff-c59e-44ce-b591-0a95d7aca3cb None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:45.145124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147719ff-c59e-44ce-b591-0a95d7aca3cb None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:45.149589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef7714-6abf-435a-9fea-5dfccc7c7d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:45.150175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef7714-6abf-435a-9fea-5dfccc7c7d41 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:45.150455 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef7714-6abf-435a-9fea-5dfccc7c7d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:45.150761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef7714-6abf-435a-9fea-5dfccc7c7d41 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:45.151391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef7714-6abf-435a-9fea-5dfccc7c7d41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:45.151690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef7714-6abf-435a-9fea-5dfccc7c7d41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:45.153261 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef7714-6abf-435a-9fea-5dfccc7c7d41 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:45.165224 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1282/2601] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:45.169386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6c74a-1d0c-4fa0-a28a-c6fbafaa7340 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:45.178866 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6c74a-1d0c-4fa0-a28a-c6fbafaa7340 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:45.183695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563edcd2-6973-48d9-8f71-544f81ec7ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:45.184285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563edcd2-6973-48d9-8f71-544f81ec7ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:45.184563 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563edcd2-6973-48d9-8f71-544f81ec7ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:45.184834 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563edcd2-6973-48d9-8f71-544f81ec7ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:45.185463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563edcd2-6973-48d9-8f71-544f81ec7ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:45.187417 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563edcd2-6973-48d9-8f71-544f81ec7ca6 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:45.284725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1320/2602] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:03:45 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:45.288841 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef7714-6abf-435a-9fea-5dfccc7c7d41 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:45.298673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef7714-6abf-435a-9fea-5dfccc7c7d41 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:45.361529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ecd84-dec2-4164-bac6-66a5fa7bd7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:45.362112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ecd84-dec2-4164-bac6-66a5fa7bd7b5 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:45.362283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ecd84-dec2-4164-bac6-66a5fa7bd7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:45.363099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ecd84-dec2-4164-bac6-66a5fa7bd7b5 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:45.363852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ecd84-dec2-4164-bac6-66a5fa7bd7b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:45.370022 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ecd84-dec2-4164-bac6-66a5fa7bd7b5 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:45.443520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1283/2603] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:03:45 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:45.447510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563edcd2-6973-48d9-8f71-544f81ec7ca6 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:45.458155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563edcd2-6973-48d9-8f71-544f81ec7ca6 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:45.463740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea143f2-a400-4d92-9899-6a57e6f37298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:45.464544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea143f2-a400-4d92-9899-6a57e6f37298 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:45.464717 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea143f2-a400-4d92-9899-6a57e6f37298 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:45.465024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea143f2-a400-4d92-9899-6a57e6f37298 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:45.465723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea143f2-a400-4d92-9899-6a57e6f37298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:45.467353 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea143f2-a400-4d92-9899-6a57e6f37298 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:45.478115 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1321/2604] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:45.485306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ecd84-dec2-4164-bac6-66a5fa7bd7b5 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:45.496591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ecd84-dec2-4164-bac6-66a5fa7bd7b5 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:45.501273 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfaff9-14b0-49f3-853f-50cef3011c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:45.501876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfaff9-14b0-49f3-853f-50cef3011c01 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:45.502156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfaff9-14b0-49f3-853f-50cef3011c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:45.502424 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfaff9-14b0-49f3-853f-50cef3011c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:45.503102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfaff9-14b0-49f3-853f-50cef3011c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:45.505492 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfaff9-14b0-49f3-853f-50cef3011c01 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:45.516339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1284/2605] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:45.520877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea143f2-a400-4d92-9899-6a57e6f37298 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:45.531203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea143f2-a400-4d92-9899-6a57e6f37298 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:45.535985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f556887-abae-4ea2-8343-15fbbd03a9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:45.536632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f556887-abae-4ea2-8343-15fbbd03a9ea admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:45.536908 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f556887-abae-4ea2-8343-15fbbd03a9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:45.537175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f556887-abae-4ea2-8343-15fbbd03a9ea admin admin] PATH_INFO: `/v3/projects/bbd57186602347128449eafc0b6a60f6/users/79f3c81a5c274e7d998c1d691a38bd65/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:45.552861 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f556887-abae-4ea2-8343-15fbbd03a9ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd57186602347128449eafc0b6a60f6, user_id=79f3c81a5c274e7d998c1d691a38bd65, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:45.555623 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f556887-abae-4ea2-8343-15fbbd03a9ea admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:45.631665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1322/2606] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:45 2022] PUT /identity/v3/projects/bbd57186602347128449eafc0b6a60f6/users/79f3c81a5c274e7d998c1d691a38bd65/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:45.696265 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c5747-ecfb-4afb-a978-c8cd9c5ced70 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:45.696579 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c5747-ecfb-4afb-a978-c8cd9c5ced70 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:45.696879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c5747-ecfb-4afb-a978-c8cd9c5ced70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:45.725966 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-278c5747-ecfb-4afb-a978-c8cd9c5ced70 None None] MFA Rules not processed for user `79f3c81a5c274e7d998c1d691a38bd65`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:45.727196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-278c5747-ecfb-4afb-a978-c8cd9c5ced70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:45.771770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:45.791108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1285/2607] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:45.795553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f556887-abae-4ea2-8343-15fbbd03a9ea admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:45.857422 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f556887-abae-4ea2-8343-15fbbd03a9ea admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:45.862387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d1fe1-9d5b-482b-8fea-d7e0f79d76e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:45.863017 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d1fe1-9d5b-482b-8fea-d7e0f79d76e3 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:45.863291 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d1fe1-9d5b-482b-8fea-d7e0f79d76e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:45.863561 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d1fe1-9d5b-482b-8fea-d7e0f79d76e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:45.864309 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d1fe1-9d5b-482b-8fea-d7e0f79d76e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:45.866041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d1fe1-9d5b-482b-8fea-d7e0f79d76e3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:45.965959 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1323/2608] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:03:45 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:03:45.969357 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c5747-ecfb-4afb-a978-c8cd9c5ced70 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:45.981349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c5747-ecfb-4afb-a978-c8cd9c5ced70 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:46.039680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402729f-471f-4148-a25e-73a10562bd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:46.040286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402729f-471f-4148-a25e-73a10562bd0f admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:46.040560 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402729f-471f-4148-a25e-73a10562bd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:46.040830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402729f-471f-4148-a25e-73a10562bd0f admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:46.041443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402729f-471f-4148-a25e-73a10562bd0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:46.043401 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402729f-471f-4148-a25e-73a10562bd0f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:46.099471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1286/2609] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:03:45 2022] POST /identity/v3/users => generated 299 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:46.103575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d1fe1-9d5b-482b-8fea-d7e0f79d76e3 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:46.114635 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d1fe1-9d5b-482b-8fea-d7e0f79d76e3 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:46.119408 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab025985-43e6-410a-987c-b62aa61a9691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:46.120000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab025985-43e6-410a-987c-b62aa61a9691 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:46.120274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab025985-43e6-410a-987c-b62aa61a9691 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:46.120544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab025985-43e6-410a-987c-b62aa61a9691 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:46.121178 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab025985-43e6-410a-987c-b62aa61a9691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:46.122817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab025985-43e6-410a-987c-b62aa61a9691 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:46.134250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1324/2610] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:46.138399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402729f-471f-4148-a25e-73a10562bd0f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:46.148556 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402729f-471f-4148-a25e-73a10562bd0f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:46.153233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da41be-1c7e-48fe-b7fc-26d0e7c32251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:46.153874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da41be-1c7e-48fe-b7fc-26d0e7c32251 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:46.154155 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da41be-1c7e-48fe-b7fc-26d0e7c32251 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:46.154430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da41be-1c7e-48fe-b7fc-26d0e7c32251 admin admin] PATH_INFO: `/v3/projects/4bf10619183043fbbd991e3dad00c4a8/users/9d8dd372674c443bbf59678e7ab7dd43/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:46.168902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da41be-1c7e-48fe-b7fc-26d0e7c32251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf10619183043fbbd991e3dad00c4a8, user_id=9d8dd372674c443bbf59678e7ab7dd43, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:46.171558 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da41be-1c7e-48fe-b7fc-26d0e7c32251 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:46.230876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1287/2611] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:46 2022] PUT /identity/v3/projects/4bf10619183043fbbd991e3dad00c4a8/users/9d8dd372674c443bbf59678e7ab7dd43/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:03:46.234819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab025985-43e6-410a-987c-b62aa61a9691 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:46.245138 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab025985-43e6-410a-987c-b62aa61a9691 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:46.273430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:46.300961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8525b-4f1a-4741-afaa-a93863aaba9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:46.301557 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8525b-4f1a-4741-afaa-a93863aaba9a admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:46.301857 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8525b-4f1a-4741-afaa-a93863aaba9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:46.302143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8525b-4f1a-4741-afaa-a93863aaba9a admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:46.302864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8525b-4f1a-4741-afaa-a93863aaba9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:46.304486 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8525b-4f1a-4741-afaa-a93863aaba9a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:46.315036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1325/2612] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:46.320017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da41be-1c7e-48fe-b7fc-26d0e7c32251 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:46.330328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da41be-1c7e-48fe-b7fc-26d0e7c32251 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:46.336264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95051066-fb0c-453c-96c8-81a841c0c527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:46.336920 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95051066-fb0c-453c-96c8-81a841c0c527 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:46.337201 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95051066-fb0c-453c-96c8-81a841c0c527 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:46.337478 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95051066-fb0c-453c-96c8-81a841c0c527 admin admin] PATH_INFO: `/v3/projects/4bf10619183043fbbd991e3dad00c4a8/users/9d8dd372674c443bbf59678e7ab7dd43/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:46.339442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95051066-fb0c-453c-96c8-81a841c0c527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf10619183043fbbd991e3dad00c4a8, user_id=9d8dd372674c443bbf59678e7ab7dd43, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:46.342081 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95051066-fb0c-453c-96c8-81a841c0c527 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:46.352919 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1288/2613] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:46 2022] PUT /identity/v3/projects/4bf10619183043fbbd991e3dad00c4a8/users/9d8dd372674c443bbf59678e7ab7dd43/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:46.363565 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e79553-eaef-47b7-a1d0-9c04ee98346b None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:46.364055 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e79553-eaef-47b7-a1d0-9c04ee98346b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:46.364377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e79553-eaef-47b7-a1d0-9c04ee98346b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:46.395669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-c8e79553-eaef-47b7-a1d0-9c04ee98346b None None] MFA Rules not processed for user `9d8dd372674c443bbf59678e7ab7dd43`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:46.396921 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c8e79553-eaef-47b7-a1d0-9c04ee98346b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:46.465748 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1326/2614] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:46.476655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95051066-fb0c-453c-96c8-81a841c0c527 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:46.478441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-95051066-fb0c-453c-96c8-81a841c0c527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:46.537422 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95051066-fb0c-453c-96c8-81a841c0c527 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:46.545621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e739995-8903-4222-bb0d-33c7f7c5304b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:46.545621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e739995-8903-4222-bb0d-33c7f7c5304b service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:46.545621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e739995-8903-4222-bb0d-33c7f7c5304b service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:46.545621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e739995-8903-4222-bb0d-33c7f7c5304b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:46.598864 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:46.602602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e739995-8903-4222-bb0d-33c7f7c5304b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:46.604039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e739995-8903-4222-bb0d-33c7f7c5304b service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:46.613625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1289/2615] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:03:50.827942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e79553-eaef-47b7-a1d0-9c04ee98346b None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:50.831187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c8e79553-eaef-47b7-a1d0-9c04ee98346b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:50.903485 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e79553-eaef-47b7-a1d0-9c04ee98346b None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:50.908806 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:50.909445 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:50.909727 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:50.910003 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] PATH_INFO: `/v3/users/42cfa3c19bfc4497865c9a1d91cfa550` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:50.911143 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=42cfa3c19bfc4497865c9a1d91cfa550)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:50.913873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:51.067399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42cfa3c19bfc4497865c9a1d91cfa550'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:51.139961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42cfa3c19bfc4497865c9a1d91cfa550'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:51.140635 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42cfa3c19bfc4497865c9a1d91cfa550'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:51.141544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42cfa3c19bfc4497865c9a1d91cfa550'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:51.143102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:51.159012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42cfa3c19bfc4497865c9a1d91cfa550'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:51.177627 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1327/2616] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:03:50 2022] DELETE /identity/v3/users/42cfa3c19bfc4497865c9a1d91cfa550 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:51.180035 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c994f-ce94-42c1-9585-8174f8aa6937 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:51.180538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c994f-ce94-42c1-9585-8174f8aa6937 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:51.180631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c994f-ce94-42c1-9585-8174f8aa6937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:51.204051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-688c994f-ce94-42c1-9585-8174f8aa6937 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:51.205593 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-688c994f-ce94-42c1-9585-8174f8aa6937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:51.274535 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1290/2617] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:51.286583 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:51.288838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:51.360729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9821c2-aa29-430d-8e90-97b48db597fc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:51.364505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dad77-da28-458e-8ca6-cafb2059ee7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:51.369140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dad77-da28-458e-8ca6-cafb2059ee7a service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:51.369140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dad77-da28-458e-8ca6-cafb2059ee7a service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:51.369140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dad77-da28-458e-8ca6-cafb2059ee7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:51.380317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665dad77-da28-458e-8ca6-cafb2059ee7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:51.382569 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665dad77-da28-458e-8ca6-cafb2059ee7a service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:51.395345 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1328/2618] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:51.806157 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c994f-ce94-42c1-9585-8174f8aa6937 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:51.807722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-688c994f-ce94-42c1-9585-8174f8aa6937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:51.812500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dad77-da28-458e-8ca6-cafb2059ee7a service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:51.814364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-665dad77-da28-458e-8ca6-cafb2059ee7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:51.825401 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c994f-ce94-42c1-9585-8174f8aa6937 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:51.830692 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:51.831602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:51.832519 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:51.833063 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] PATH_INFO: `/v3/projects/68f25a496fe247088fbe40c155b109c6` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:51.834561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f25a496fe247088fbe40c155b109c6)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:51.836600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:51.882288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dad77-da28-458e-8ca6-cafb2059ee7a service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:51.886927 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106fad0-4b97-4fbe-bde1-8fe36f858c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:51.887729 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106fad0-4b97-4fbe-bde1-8fe36f858c0c service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:51.888189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106fad0-4b97-4fbe-bde1-8fe36f858c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:51.888635 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106fad0-4b97-4fbe-bde1-8fe36f858c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:51.890415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-3106fad0-4b97-4fbe-bde1-8fe36f858c0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:51.930858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f25a496fe247088fbe40c155b109c6', 'request_id': 'req-9fd395c0-6655-4a55-997a-74af91632df7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:51.937401 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f25a496fe247088fbe40c155b109c6', 'request_id': 'req-9fd395c0-6655-4a55-997a-74af91632df7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:51.954968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106fad0-4b97-4fbe-bde1-8fe36f858c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:51.956426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106fad0-4b97-4fbe-bde1-8fe36f858c0c service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:51.966190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1329/2619] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:03:52.020036 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f25a496fe247088fbe40c155b109c6', 'request_id': 'req-9fd395c0-6655-4a55-997a-74af91632df7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:52.034852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] The token cache is being invalidate because project 68f25a496fe247088fbe40c155b109c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:03:52.035264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:52.036087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:52.037488 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1291/2620] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:03:51 2022] DELETE /identity/v3/projects/68f25a496fe247088fbe40c155b109c6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:52.042421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106fad0-4b97-4fbe-bde1-8fe36f858c0c service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:52.052856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106fad0-4b97-4fbe-bde1-8fe36f858c0c service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:52.072987 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:52.110437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4188269-7cc5-4e3d-931f-3f8a16469cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:52.111089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4188269-7cc5-4e3d-931f-3f8a16469cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:52.111363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4188269-7cc5-4e3d-931f-3f8a16469cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:52.112410 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4188269-7cc5-4e3d-931f-3f8a16469cc9 admin admin] PATH_INFO: `/v3/users/6948ebc008c9483d9aac480644f4ff4b` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:52.112864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4188269-7cc5-4e3d-931f-3f8a16469cc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6948ebc008c9483d9aac480644f4ff4b)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:52.114809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4188269-7cc5-4e3d-931f-3f8a16469cc9 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:52.325985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c4188269-7cc5-4e3d-931f-3f8a16469cc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6948ebc008c9483d9aac480644f4ff4b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:52.420122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c4188269-7cc5-4e3d-931f-3f8a16469cc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6948ebc008c9483d9aac480644f4ff4b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:52.429762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c4188269-7cc5-4e3d-931f-3f8a16469cc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6948ebc008c9483d9aac480644f4ff4b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:52.429762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c4188269-7cc5-4e3d-931f-3f8a16469cc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6948ebc008c9483d9aac480644f4ff4b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:52.434690 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c4188269-7cc5-4e3d-931f-3f8a16469cc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6948ebc008c9483d9aac480644f4ff4b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:52.448688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1330/2621] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:03:52 2022] DELETE /identity/v3/users/6948ebc008c9483d9aac480644f4ff4b => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:52.876697 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:52.878413 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:52.892504 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd395c0-6655-4a55-997a-74af91632df7 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:52.952066 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480a804-42f1-48eb-9472-48410f9e15b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:52.952693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a804-42f1-48eb-9472-48410f9e15b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:52.952972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a804-42f1-48eb-9472-48410f9e15b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:52.953243 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a804-42f1-48eb-9472-48410f9e15b0 admin admin] PATH_INFO: `/v3/projects/2faf81e109644e06a89dd7160e6fb7c6` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:52.954341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480a804-42f1-48eb-9472-48410f9e15b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2faf81e109644e06a89dd7160e6fb7c6)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:52.956255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480a804-42f1-48eb-9472-48410f9e15b0 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:52.982321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:53.042867 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b480a804-42f1-48eb-9472-48410f9e15b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2faf81e109644e06a89dd7160e6fb7c6', 'request_id': 'req-b480a804-42f1-48eb-9472-48410f9e15b0'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:53.049124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b480a804-42f1-48eb-9472-48410f9e15b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2faf81e109644e06a89dd7160e6fb7c6', 'request_id': 'req-b480a804-42f1-48eb-9472-48410f9e15b0'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:53.134229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b480a804-42f1-48eb-9472-48410f9e15b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2faf81e109644e06a89dd7160e6fb7c6', 'request_id': 'req-b480a804-42f1-48eb-9472-48410f9e15b0'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:53.145186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b480a804-42f1-48eb-9472-48410f9e15b0 admin admin] The token cache is being invalidate because project 2faf81e109644e06a89dd7160e6fb7c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:03:53.145650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b480a804-42f1-48eb-9472-48410f9e15b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:53.146842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b480a804-42f1-48eb-9472-48410f9e15b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:53.152081 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1292/2622] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:03:52 2022] DELETE /identity/v3/projects/2faf81e109644e06a89dd7160e6fb7c6 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:53.154055 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4db358-4c89-4aea-a100-62646916d349 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:53.154521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4db358-4c89-4aea-a100-62646916d349 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:53.154632 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4db358-4c89-4aea-a100-62646916d349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:53.174871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-cd4db358-4c89-4aea-a100-62646916d349 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:53.176150 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-cd4db358-4c89-4aea-a100-62646916d349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:53.235510 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1331/2623] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:53.243575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8558b-6a63-4719-9c5b-35c9545e2d8a None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:53.243868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8558b-6a63-4719-9c5b-35c9545e2d8a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:53.244174 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8558b-6a63-4719-9c5b-35c9545e2d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:53.264256 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-ddd8558b-6a63-4719-9c5b-35c9545e2d8a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:53.265461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-ddd8558b-6a63-4719-9c5b-35c9545e2d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:53.273962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1293/2624] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:53.278304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4db358-4c89-4aea-a100-62646916d349 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:53.288112 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4db358-4c89-4aea-a100-62646916d349 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:53.293322 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ddceb-9fba-463b-911e-36129b31f873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:53.293929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ddceb-9fba-463b-911e-36129b31f873 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:53.294209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ddceb-9fba-463b-911e-36129b31f873 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:53.294508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ddceb-9fba-463b-911e-36129b31f873 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:53.295172 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ddceb-9fba-463b-911e-36129b31f873 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:53.295476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ddceb-9fba-463b-911e-36129b31f873 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:53.297123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ddceb-9fba-463b-911e-36129b31f873 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:53.309152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1332/2625] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:53.313395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8558b-6a63-4719-9c5b-35c9545e2d8a None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:53.323397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8558b-6a63-4719-9c5b-35c9545e2d8a None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:53.328089 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e83863-7777-4b7c-a888-3ed9e2e47555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:53.328675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e83863-7777-4b7c-a888-3ed9e2e47555 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:53.328950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e83863-7777-4b7c-a888-3ed9e2e47555 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:53.329220 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e83863-7777-4b7c-a888-3ed9e2e47555 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:53.329876 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e83863-7777-4b7c-a888-3ed9e2e47555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:53.331773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e83863-7777-4b7c-a888-3ed9e2e47555 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:53.451897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1294/2626] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:03:53 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:53.455697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ddceb-9fba-463b-911e-36129b31f873 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:53.466871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ddceb-9fba-463b-911e-36129b31f873 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:53.495247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:53.525443 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37701d8-341e-4db8-bb14-2d2b2b855ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:53.529566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37701d8-341e-4db8-bb14-2d2b2b855ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:53.529566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37701d8-341e-4db8-bb14-2d2b2b855ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:53.529566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37701d8-341e-4db8-bb14-2d2b2b855ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:53.529566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37701d8-341e-4db8-bb14-2d2b2b855ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:53.529566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37701d8-341e-4db8-bb14-2d2b2b855ea6 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:53.637940 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1333/2627] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:03:53 2022] POST /identity/v3/users => generated 291 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:53.641861 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e83863-7777-4b7c-a888-3ed9e2e47555 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:53.652405 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e83863-7777-4b7c-a888-3ed9e2e47555 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:53.657043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0a2f6-6af7-4c78-85a8-19a5b0028fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:53.657646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0a2f6-6af7-4c78-85a8-19a5b0028fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:53.657930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0a2f6-6af7-4c78-85a8-19a5b0028fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:53.658208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0a2f6-6af7-4c78-85a8-19a5b0028fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:53.658889 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0a2f6-6af7-4c78-85a8-19a5b0028fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:53.660474 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0a2f6-6af7-4c78-85a8-19a5b0028fd9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:53.670447 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1295/2628] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:53.674522 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37701d8-341e-4db8-bb14-2d2b2b855ea6 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:53.685317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37701d8-341e-4db8-bb14-2d2b2b855ea6 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:53.689890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b557e-e10d-43a6-bdfb-a37b3e5649f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:53.690515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b557e-e10d-43a6-bdfb-a37b3e5649f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:53.690787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b557e-e10d-43a6-bdfb-a37b3e5649f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:53.691203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b557e-e10d-43a6-bdfb-a37b3e5649f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:53.692330 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b557e-e10d-43a6-bdfb-a37b3e5649f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:53.695906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b557e-e10d-43a6-bdfb-a37b3e5649f3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:53.712599 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1334/2629] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:53.713567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0a2f6-6af7-4c78-85a8-19a5b0028fd9 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:53.723557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0a2f6-6af7-4c78-85a8-19a5b0028fd9 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:53.728329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915b47a-053a-4346-9fdc-f7f3a613d492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:53.728995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915b47a-053a-4346-9fdc-f7f3a613d492 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:53.729273 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915b47a-053a-4346-9fdc-f7f3a613d492 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:53.729542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915b47a-053a-4346-9fdc-f7f3a613d492 admin admin] PATH_INFO: `/v3/projects/f6ce36bd76ee47adb0a3920649c9b982/users/0e30d15ee3334772b7ab2e1e8341abbb/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:53.745072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915b47a-053a-4346-9fdc-f7f3a613d492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ce36bd76ee47adb0a3920649c9b982, user_id=0e30d15ee3334772b7ab2e1e8341abbb, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:53.747720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915b47a-053a-4346-9fdc-f7f3a613d492 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:53.855262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1296/2630] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:53 2022] PUT /identity/v3/projects/f6ce36bd76ee47adb0a3920649c9b982/users/0e30d15ee3334772b7ab2e1e8341abbb/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:53.865506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962d8e9-970a-40e7-b7f2-0e82361ea7d0 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:53.866028 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962d8e9-970a-40e7-b7f2-0e82361ea7d0 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:53.866450 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962d8e9-970a-40e7-b7f2-0e82361ea7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:53.900925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-f962d8e9-970a-40e7-b7f2-0e82361ea7d0 None None] MFA Rules not processed for user `0e30d15ee3334772b7ab2e1e8341abbb`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:53.902256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f962d8e9-970a-40e7-b7f2-0e82361ea7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:53.972972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1335/2631] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:53.977956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915b47a-053a-4346-9fdc-f7f3a613d492 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:54.051896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915b47a-053a-4346-9fdc-f7f3a613d492 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:54.056557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ae2ee-6221-40fc-b5ad-9fd578ab0f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:54.059539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae2ee-6221-40fc-b5ad-9fd578ab0f81 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:54.059539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae2ee-6221-40fc-b5ad-9fd578ab0f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:54.059539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae2ee-6221-40fc-b5ad-9fd578ab0f81 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:54.059539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ae2ee-6221-40fc-b5ad-9fd578ab0f81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:54.060212 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ae2ee-6221-40fc-b5ad-9fd578ab0f81 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:54.146807 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1297/2632] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:03:53 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:54.151922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962d8e9-970a-40e7-b7f2-0e82361ea7d0 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:54.162694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962d8e9-970a-40e7-b7f2-0e82361ea7d0 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:54.220281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d494e6-68ce-4e60-882e-15dbdfae7228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:54.220918 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d494e6-68ce-4e60-882e-15dbdfae7228 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:54.221200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d494e6-68ce-4e60-882e-15dbdfae7228 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:54.221497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d494e6-68ce-4e60-882e-15dbdfae7228 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:54.222145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d494e6-68ce-4e60-882e-15dbdfae7228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:54.224118 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d494e6-68ce-4e60-882e-15dbdfae7228 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:54.300321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1336/2633] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:03:54 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:54.305169 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ae2ee-6221-40fc-b5ad-9fd578ab0f81 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:54.317202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ae2ee-6221-40fc-b5ad-9fd578ab0f81 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:54.323630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14113171-1b63-4440-be25-560f3f5f6f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:54.336616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14113171-1b63-4440-be25-560f3f5f6f1f admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:54.336616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14113171-1b63-4440-be25-560f3f5f6f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:54.336616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14113171-1b63-4440-be25-560f3f5f6f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:54.336616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14113171-1b63-4440-be25-560f3f5f6f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:54.336616 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14113171-1b63-4440-be25-560f3f5f6f1f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:54.338581 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1298/2634] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:54.342844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d494e6-68ce-4e60-882e-15dbdfae7228 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:54.353824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d494e6-68ce-4e60-882e-15dbdfae7228 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:54.358827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe532b52-59be-45b8-a458-5de9a90e389f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:54.359485 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe532b52-59be-45b8-a458-5de9a90e389f admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:54.359872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe532b52-59be-45b8-a458-5de9a90e389f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:54.360037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe532b52-59be-45b8-a458-5de9a90e389f admin admin] PATH_INFO: `/v3/projects/043b41bc63364e669b0714d4ae6675a6/users/30ba55d984904902b6dfb95e46fa5b7b/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:54.375779 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe532b52-59be-45b8-a458-5de9a90e389f admin admin] RBAC: Authorizing `identity:create_grant(project_id=043b41bc63364e669b0714d4ae6675a6, user_id=30ba55d984904902b6dfb95e46fa5b7b, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:54.378598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe532b52-59be-45b8-a458-5de9a90e389f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:54.431722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1337/2635] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:54 2022] PUT /identity/v3/projects/043b41bc63364e669b0714d4ae6675a6/users/30ba55d984904902b6dfb95e46fa5b7b/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:54.436026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14113171-1b63-4440-be25-560f3f5f6f1f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:54.447664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14113171-1b63-4440-be25-560f3f5f6f1f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:54.461480 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:54.505871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c55cb-ffa6-482c-84ef-54f7ae863ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:54.506376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c55cb-ffa6-482c-84ef-54f7ae863ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:54.506684 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c55cb-ffa6-482c-84ef-54f7ae863ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:54.507520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c55cb-ffa6-482c-84ef-54f7ae863ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:54.508147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c55cb-ffa6-482c-84ef-54f7ae863ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:54.509809 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c55cb-ffa6-482c-84ef-54f7ae863ba1 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:54.520271 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1299/2636] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:03:54.524636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe532b52-59be-45b8-a458-5de9a90e389f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:54.531372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:54.535291 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe532b52-59be-45b8-a458-5de9a90e389f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:54.540674 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7efba0-1bfe-45d1-b158-dc051c9897dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:54.543429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7efba0-1bfe-45d1-b158-dc051c9897dc admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:54.543429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7efba0-1bfe-45d1-b158-dc051c9897dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:54.543429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7efba0-1bfe-45d1-b158-dc051c9897dc admin admin] PATH_INFO: `/v3/projects/043b41bc63364e669b0714d4ae6675a6/users/30ba55d984904902b6dfb95e46fa5b7b/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:54.543736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7efba0-1bfe-45d1-b158-dc051c9897dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=043b41bc63364e669b0714d4ae6675a6, user_id=30ba55d984904902b6dfb95e46fa5b7b, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:54.546358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7efba0-1bfe-45d1-b158-dc051c9897dc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:54.557134 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1338/2637] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:03:54 2022] PUT /identity/v3/projects/043b41bc63364e669b0714d4ae6675a6/users/30ba55d984904902b6dfb95e46fa5b7b/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:54.568281 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cae73-87ef-43d3-a3ed-0c8d73c9a2f9 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:54.568570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cae73-87ef-43d3-a3ed-0c8d73c9a2f9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:54.568846 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cae73-87ef-43d3-a3ed-0c8d73c9a2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:54.598200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-149cae73-87ef-43d3-a3ed-0c8d73c9a2f9 None None] MFA Rules not processed for user `30ba55d984904902b6dfb95e46fa5b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:54.599775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-149cae73-87ef-43d3-a3ed-0c8d73c9a2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:54.670145 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1300/2638] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:54.683292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7efba0-1bfe-45d1-b158-dc051c9897dc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:54.685087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8c7efba0-1bfe-45d1-b158-dc051c9897dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:54.743407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7efba0-1bfe-45d1-b158-dc051c9897dc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:54.747064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983646a0-ee43-4197-8019-6304caae6da9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:54.747660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983646a0-ee43-4197-8019-6304caae6da9 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:54.747937 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983646a0-ee43-4197-8019-6304caae6da9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:54.748205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983646a0-ee43-4197-8019-6304caae6da9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:54.757767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983646a0-ee43-4197-8019-6304caae6da9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:54.759148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983646a0-ee43-4197-8019-6304caae6da9 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:54.768669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1339/2639] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:55.752991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cae73-87ef-43d3-a3ed-0c8d73c9a2f9 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:55.762297 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cae73-87ef-43d3-a3ed-0c8d73c9a2f9 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:55.766942 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388aeeef-6614-4a0b-baf3-709928d2b628 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30ba55d984904902b6dfb95e46fa5b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '043b41bc63364e669b0714d4ae6675a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:55.767509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388aeeef-6614-4a0b-baf3-709928d2b628 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:55.767803 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388aeeef-6614-4a0b-baf3-709928d2b628 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:55.768078 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388aeeef-6614-4a0b-baf3-709928d2b628 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:55.768720 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388aeeef-6614-4a0b-baf3-709928d2b628 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:55.769019 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388aeeef-6614-4a0b-baf3-709928d2b628 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:55.770687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388aeeef-6614-4a0b-baf3-709928d2b628 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:55.783185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1301/2640] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:55.787440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983646a0-ee43-4197-8019-6304caae6da9 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:55.796965 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983646a0-ee43-4197-8019-6304caae6da9 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:55.801762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480274c-f368-48e9-9196-f9c644057056 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30ba55d984904902b6dfb95e46fa5b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '043b41bc63364e669b0714d4ae6675a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:55.802424 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480274c-f368-48e9-9196-f9c644057056 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:55.808247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480274c-f368-48e9-9196-f9c644057056 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:55.808247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480274c-f368-48e9-9196-f9c644057056 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:55.808247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480274c-f368-48e9-9196-f9c644057056 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:55.808247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480274c-f368-48e9-9196-f9c644057056 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:55.883933 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1340/2641] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:03:55 2022] POST /identity/v3/projects => generated 355 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:03:55.888072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388aeeef-6614-4a0b-baf3-709928d2b628 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:55.898657 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388aeeef-6614-4a0b-baf3-709928d2b628 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:55.955005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e1c29-372b-405f-bdbf-609d7e27fdc6 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30ba55d984904902b6dfb95e46fa5b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '043b41bc63364e669b0714d4ae6675a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:55.955617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e1c29-372b-405f-bdbf-609d7e27fdc6 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:55.955905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e1c29-372b-405f-bdbf-609d7e27fdc6 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:55.956186 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e1c29-372b-405f-bdbf-609d7e27fdc6 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:55.957124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e1c29-372b-405f-bdbf-609d7e27fdc6 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:55.957439 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e1c29-372b-405f-bdbf-609d7e27fdc6 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:55.959057 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e1c29-372b-405f-bdbf-609d7e27fdc6 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:55.971933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1302/2642] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:56.008984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480274c-f368-48e9-9196-f9c644057056 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:56.023805 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480274c-f368-48e9-9196-f9c644057056 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:56.028833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f0c17-393f-43ba-bad5-c1a70ceedb70 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30ba55d984904902b6dfb95e46fa5b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '043b41bc63364e669b0714d4ae6675a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:56.029455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f0c17-393f-43ba-bad5-c1a70ceedb70 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:56.029734 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f0c17-393f-43ba-bad5-c1a70ceedb70 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:56.030053 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f0c17-393f-43ba-bad5-c1a70ceedb70 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] PATH_INFO: `/v3/projects/640f8518a32f40cfa8beff9c295b016f` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:56.031177 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f0c17-393f-43ba-bad5-c1a70ceedb70 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: Authorizing `identity:delete_project(project_id=640f8518a32f40cfa8beff9c295b016f)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:56.033085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f0c17-393f-43ba-bad5-c1a70ceedb70 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:56.164364 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a22f0c17-393f-43ba-bad5-c1a70ceedb70 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '640f8518a32f40cfa8beff9c295b016f', 'request_id': 'req-a22f0c17-393f-43ba-bad5-c1a70ceedb70'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:56.171074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a22f0c17-393f-43ba-bad5-c1a70ceedb70 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '640f8518a32f40cfa8beff9c295b016f', 'request_id': 'req-a22f0c17-393f-43ba-bad5-c1a70ceedb70'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:56.242499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a22f0c17-393f-43ba-bad5-c1a70ceedb70 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '640f8518a32f40cfa8beff9c295b016f', 'request_id': 'req-a22f0c17-393f-43ba-bad5-c1a70ceedb70'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:56.253175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a22f0c17-393f-43ba-bad5-c1a70ceedb70 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] The token cache is being invalidate because project 640f8518a32f40cfa8beff9c295b016f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:03:56.253447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a22f0c17-393f-43ba-bad5-c1a70ceedb70 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:56.254092 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a22f0c17-393f-43ba-bad5-c1a70ceedb70 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:56.255644 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1341/2643] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:03:56 2022] DELETE /identity/v3/projects/640f8518a32f40cfa8beff9c295b016f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:56.898438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e1c29-372b-405f-bdbf-609d7e27fdc6 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:56.900285 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-519e1c29-372b-405f-bdbf-609d7e27fdc6 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:56.922140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:56.966953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e1c29-372b-405f-bdbf-609d7e27fdc6 tempest-VolumeQuotaClassesTest-711478549 tempest-VolumeQuotaClassesTest-711478549-project-admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:57.028672 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:57.029300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:57.029578 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:57.029847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] PATH_INFO: `/v3/users/0e30d15ee3334772b7ab2e1e8341abbb` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:57.031196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e30d15ee3334772b7ab2e1e8341abbb)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:57.033095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:57.256824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e30d15ee3334772b7ab2e1e8341abbb'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:57.333847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e30d15ee3334772b7ab2e1e8341abbb'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:57.334552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e30d15ee3334772b7ab2e1e8341abbb'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:57.335965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e30d15ee3334772b7ab2e1e8341abbb'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:57.343618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e30d15ee3334772b7ab2e1e8341abbb'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:57.363020 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1303/2644] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:03:56 2022] DELETE /identity/v3/users/0e30d15ee3334772b7ab2e1e8341abbb => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:57.367982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83749f07-a3fb-4619-ab5e-209cb225d64d None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:57.368750 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83749f07-a3fb-4619-ab5e-209cb225d64d None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:57.368750 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83749f07-a3fb-4619-ab5e-209cb225d64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:57.388156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-83749f07-a3fb-4619-ab5e-209cb225d64d None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:57.389360 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-83749f07-a3fb-4619-ab5e-209cb225d64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:57.438619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:57.450690 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1342/2645] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:03:57.461343 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:57.462797 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:57.514344 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:57.519119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06722052-8904-4fe2-8e8a-5f2f9ad8ae54 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:57.523266 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b42bea-f3e4-4334-ae8e-66224b13a26e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:57.523860 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b42bea-f3e4-4334-ae8e-66224b13a26e service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:57.528974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b42bea-f3e4-4334-ae8e-66224b13a26e service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:57.528974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b42bea-f3e4-4334-ae8e-66224b13a26e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:57.533370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b42bea-f3e4-4334-ae8e-66224b13a26e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:57.534835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b42bea-f3e4-4334-ae8e-66224b13a26e service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:57.545299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1304/2646] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:03:57.860539 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83749f07-a3fb-4619-ab5e-209cb225d64d None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:57.862011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-83749f07-a3fb-4619-ab5e-209cb225d64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:57.874159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83749f07-a3fb-4619-ab5e-209cb225d64d None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:57.878762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e3a63-8431-4a65-834a-8e78f1f2e68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:57.879386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e3a63-8431-4a65-834a-8e78f1f2e68a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:57.879671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e3a63-8431-4a65-834a-8e78f1f2e68a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:57.879947 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e3a63-8431-4a65-834a-8e78f1f2e68a admin admin] PATH_INFO: `/v3/users/79f3c81a5c274e7d998c1d691a38bd65` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:57.881073 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e3a63-8431-4a65-834a-8e78f1f2e68a admin admin] RBAC: Authorizing `identity:delete_user(user_id=79f3c81a5c274e7d998c1d691a38bd65)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:57.883527 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e3a63-8431-4a65-834a-8e78f1f2e68a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:57.953625 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b42bea-f3e4-4334-ae8e-66224b13a26e service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:57.955230 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-91b42bea-f3e4-4334-ae8e-66224b13a26e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:57.969673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b42bea-f3e4-4334-ae8e-66224b13a26e service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:57.974868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:57.978338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:57.978338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:57.978338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] PATH_INFO: `/v3/projects/f6ce36bd76ee47adb0a3920649c9b982` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:57.978338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ce36bd76ee47adb0a3920649c9b982)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:57.979460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:58.097060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-991e3a63-8431-4a65-834a-8e78f1f2e68a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79f3c81a5c274e7d998c1d691a38bd65'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.106291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ce36bd76ee47adb0a3920649c9b982', 'request_id': 'req-d8642a51-878e-4560-89d1-aed902c65864'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.112971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ce36bd76ee47adb0a3920649c9b982', 'request_id': 'req-d8642a51-878e-4560-89d1-aed902c65864'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.162562 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-991e3a63-8431-4a65-834a-8e78f1f2e68a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79f3c81a5c274e7d998c1d691a38bd65'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.163831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-991e3a63-8431-4a65-834a-8e78f1f2e68a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79f3c81a5c274e7d998c1d691a38bd65'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.164438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-991e3a63-8431-4a65-834a-8e78f1f2e68a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79f3c81a5c274e7d998c1d691a38bd65'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.177473 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-991e3a63-8431-4a65-834a-8e78f1f2e68a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79f3c81a5c274e7d998c1d691a38bd65'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.190022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1343/2647] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:03:57 2022] DELETE /identity/v3/users/79f3c81a5c274e7d998c1d691a38bd65 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:58.194662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927ac10-acb1-4938-9153-83e293ff6117 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:58.195128 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927ac10-acb1-4938-9153-83e293ff6117 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:58.195407 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927ac10-acb1-4938-9153-83e293ff6117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:58.202454 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ce36bd76ee47adb0a3920649c9b982', 'request_id': 'req-d8642a51-878e-4560-89d1-aed902c65864'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.215574 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] The token cache is being invalidate because project f6ce36bd76ee47adb0a3920649c9b982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:03:58.215864 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.216496 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.217010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-5927ac10-acb1-4938-9153-83e293ff6117 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:58.218079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1305/2648] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:03:57 2022] DELETE /identity/v3/projects/f6ce36bd76ee47adb0a3920649c9b982 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:03:58.218243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5927ac10-acb1-4938-9153-83e293ff6117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:58.222152 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:58.223643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:58.296245 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8642a51-878e-4560-89d1-aed902c65864 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:58.360619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:58.361170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:58.361485 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:58.361785 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] PATH_INFO: `/v3/users/30ba55d984904902b6dfb95e46fa5b7b` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:58.362944 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ba55d984904902b6dfb95e46fa5b7b)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:58.364861 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:58.388366 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:58.403153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1344/2649] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:03:58.412733 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927ac10-acb1-4938-9153-83e293ff6117 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:58.414095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5927ac10-acb1-4938-9153-83e293ff6117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:58.469614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927ac10-acb1-4938-9153-83e293ff6117 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:58.473216 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca234e76-5c73-4e3f-82a0-4574b783ae55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:58.473808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca234e76-5c73-4e3f-82a0-4574b783ae55 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:58.474085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca234e76-5c73-4e3f-82a0-4574b783ae55 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:58.474358 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca234e76-5c73-4e3f-82a0-4574b783ae55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:58.484059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca234e76-5c73-4e3f-82a0-4574b783ae55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:58.485449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca234e76-5c73-4e3f-82a0-4574b783ae55 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:58.495555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1345/2650] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:03:58.590947 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ba55d984904902b6dfb95e46fa5b7b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.671151 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ba55d984904902b6dfb95e46fa5b7b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.671767 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ba55d984904902b6dfb95e46fa5b7b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.672375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ba55d984904902b6dfb95e46fa5b7b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.680460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ba55d984904902b6dfb95e46fa5b7b'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:58.699563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1306/2651] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:03:58 2022] DELETE /identity/v3/users/30ba55d984904902b6dfb95e46fa5b7b => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:03:58.907111 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca234e76-5c73-4e3f-82a0-4574b783ae55 service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:58.909130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ca234e76-5c73-4e3f-82a0-4574b783ae55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:58.926916 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca234e76-5c73-4e3f-82a0-4574b783ae55 service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:58.935863 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:58.988345 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:58.988977 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:58.989250 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:58.989518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] PATH_INFO: `/v3/projects/bbd57186602347128449eafc0b6a60f6` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:58.990671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd57186602347128449eafc0b6a60f6)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:58.993015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:59.095429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd57186602347128449eafc0b6a60f6', 'request_id': 'req-b61e80a1-678e-41f0-8833-6f083da7ac67'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.102690 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd57186602347128449eafc0b6a60f6', 'request_id': 'req-b61e80a1-678e-41f0-8833-6f083da7ac67'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.120218 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:59.121832 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:59.136337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112c010-6517-4fa1-89dc-9272fc92708f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:59.196794 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0277f4f-0da3-4849-8517-2226e9355f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:59.197015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd57186602347128449eafc0b6a60f6', 'request_id': 'req-b61e80a1-678e-41f0-8833-6f083da7ac67'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.197354 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0277f4f-0da3-4849-8517-2226e9355f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:59.197631 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0277f4f-0da3-4849-8517-2226e9355f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:59.197903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0277f4f-0da3-4849-8517-2226e9355f0d admin admin] PATH_INFO: `/v3/projects/043b41bc63364e669b0714d4ae6675a6` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:59.199205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0277f4f-0da3-4849-8517-2226e9355f0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=043b41bc63364e669b0714d4ae6675a6)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:59.200997 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0277f4f-0da3-4849-8517-2226e9355f0d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:59.208156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] The token cache is being invalidate because project bbd57186602347128449eafc0b6a60f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:03:59.208447 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.209089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.211171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1346/2652] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:03:58 2022] DELETE /identity/v3/projects/bbd57186602347128449eafc0b6a60f6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:59.215011 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:59.216367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:59.229480 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e80a1-678e-41f0-8833-6f083da7ac67 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:59.234180 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f55254-de8d-4b11-a7d7-13613dc4914a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:59.234835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f55254-de8d-4b11-a7d7-13613dc4914a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:59.235113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f55254-de8d-4b11-a7d7-13613dc4914a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:59.235385 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f55254-de8d-4b11-a7d7-13613dc4914a admin admin] PATH_INFO: `/v3/users/9d8dd372674c443bbf59678e7ab7dd43` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:59.236571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f55254-de8d-4b11-a7d7-13613dc4914a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d8dd372674c443bbf59678e7ab7dd43)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:59.238514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f55254-de8d-4b11-a7d7-13613dc4914a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:59.340913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b0277f4f-0da3-4849-8517-2226e9355f0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '043b41bc63364e669b0714d4ae6675a6', 'request_id': 'req-b0277f4f-0da3-4849-8517-2226e9355f0d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.347189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b0277f4f-0da3-4849-8517-2226e9355f0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '043b41bc63364e669b0714d4ae6675a6', 'request_id': 'req-b0277f4f-0da3-4849-8517-2226e9355f0d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.479687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b0277f4f-0da3-4849-8517-2226e9355f0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '043b41bc63364e669b0714d4ae6675a6', 'request_id': 'req-b0277f4f-0da3-4849-8517-2226e9355f0d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.491334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b0277f4f-0da3-4849-8517-2226e9355f0d admin admin] The token cache is being invalidate because project 043b41bc63364e669b0714d4ae6675a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:03:59.491597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-77f55254-de8d-4b11-a7d7-13613dc4914a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d8dd372674c443bbf59678e7ab7dd43'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.492047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b0277f4f-0da3-4849-8517-2226e9355f0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.492355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b0277f4f-0da3-4849-8517-2226e9355f0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.495232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1307/2653] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:03:59 2022] DELETE /identity/v3/projects/043b41bc63364e669b0714d4ae6675a6 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:03:59.499834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e26270-2a0c-4598-9f78-1948754a3fb1 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:59.500121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e26270-2a0c-4598-9f78-1948754a3fb1 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:59.500396 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e26270-2a0c-4598-9f78-1948754a3fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:59.520460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-33e26270-2a0c-4598-9f78-1948754a3fb1 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:59.521700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-33e26270-2a0c-4598-9f78-1948754a3fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:59.547536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:59.562864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-77f55254-de8d-4b11-a7d7-13613dc4914a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d8dd372674c443bbf59678e7ab7dd43'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.563490 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-77f55254-de8d-4b11-a7d7-13613dc4914a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d8dd372674c443bbf59678e7ab7dd43'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.564109 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-77f55254-de8d-4b11-a7d7-13613dc4914a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d8dd372674c443bbf59678e7ab7dd43'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.577979 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-77f55254-de8d-4b11-a7d7-13613dc4914a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d8dd372674c443bbf59678e7ab7dd43'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:03:59.582402 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1308/2654] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:03:59.591285 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4b154-c1c2-457e-9d9d-3e7c7387b793 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:59.591575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4b154-c1c2-457e-9d9d-3e7c7387b793 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:59.591855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4b154-c1c2-457e-9d9d-3e7c7387b793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:59.592584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1347/2655] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:03:59 2022] DELETE /identity/v3/users/9d8dd372674c443bbf59678e7ab7dd43 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:03:59.613711 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-0df4b154-c1c2-457e-9d9d-3e7c7387b793 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:03:59.615022 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0df4b154-c1c2-457e-9d9d-3e7c7387b793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:03:59.678778 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1309/2656] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:03:59.683763 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f55254-de8d-4b11-a7d7-13613dc4914a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:59.693497 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f55254-de8d-4b11-a7d7-13613dc4914a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:59.698657 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f893105-345b-4cd9-bcf2-be71606291c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:59.699254 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f893105-345b-4cd9-bcf2-be71606291c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:59.699532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f893105-345b-4cd9-bcf2-be71606291c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:59.699800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f893105-345b-4cd9-bcf2-be71606291c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:59.700439 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f893105-345b-4cd9-bcf2-be71606291c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:03:59.700737 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f893105-345b-4cd9-bcf2-be71606291c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:59.702309 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f893105-345b-4cd9-bcf2-be71606291c5 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:59.712043 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:03:59.715838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1348/2657] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:03:59.720159 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4b154-c1c2-457e-9d9d-3e7c7387b793 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:59.729974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4b154-c1c2-457e-9d9d-3e7c7387b793 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:59.734436 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881a7ed-21a2-4c65-aab3-954dfdf6b32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:59.735054 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881a7ed-21a2-4c65-aab3-954dfdf6b32b admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:59.735332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881a7ed-21a2-4c65-aab3-954dfdf6b32b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:59.735598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881a7ed-21a2-4c65-aab3-954dfdf6b32b admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:59.736231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881a7ed-21a2-4c65-aab3-954dfdf6b32b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:59.738423 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881a7ed-21a2-4c65-aab3-954dfdf6b32b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:03:59.859669 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1310/2658] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:03:59 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:03:59.860812 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f893105-345b-4cd9-bcf2-be71606291c5 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:03:59.871199 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f893105-345b-4cd9-bcf2-be71606291c5 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:03:59.930316 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5a3dd-2f3a-473e-8018-cdd9983b95bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:03:59.931041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5a3dd-2f3a-473e-8018-cdd9983b95bf admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:03:59.931332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5a3dd-2f3a-473e-8018-cdd9983b95bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:03:59.931601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5a3dd-2f3a-473e-8018-cdd9983b95bf admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:03:59.932288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5a3dd-2f3a-473e-8018-cdd9983b95bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:03:59.934166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5a3dd-2f3a-473e-8018-cdd9983b95bf admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.010415 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881a7ed-21a2-4c65-aab3-954dfdf6b32b admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.012177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9881a7ed-21a2-4c65-aab3-954dfdf6b32b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:00.025463 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881a7ed-21a2-4c65-aab3-954dfdf6b32b admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.030107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79c5af-f517-4a51-a4f5-6eb73115d72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.030770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79c5af-f517-4a51-a4f5-6eb73115d72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.031047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79c5af-f517-4a51-a4f5-6eb73115d72c admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.031314 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79c5af-f517-4a51-a4f5-6eb73115d72c admin admin] PATH_INFO: `/v3/projects/4bf10619183043fbbd991e3dad00c4a8` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.032433 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79c5af-f517-4a51-a4f5-6eb73115d72c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bf10619183043fbbd991e3dad00c4a8)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.034323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79c5af-f517-4a51-a4f5-6eb73115d72c admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.052022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1349/2659] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:03:59 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:04:00.053040 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5a3dd-2f3a-473e-8018-cdd9983b95bf admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.062776 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5a3dd-2f3a-473e-8018-cdd9983b95bf admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.067545 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8ad6b-a7ec-4adb-bacf-ceb2ecf63e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.068133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8ad6b-a7ec-4adb-bacf-ceb2ecf63e35 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.068410 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8ad6b-a7ec-4adb-bacf-ceb2ecf63e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.068678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8ad6b-a7ec-4adb-bacf-ceb2ecf63e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.069310 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8ad6b-a7ec-4adb-bacf-ceb2ecf63e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.071608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8ad6b-a7ec-4adb-bacf-ceb2ecf63e35 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.082500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1350/2660] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:04:00.087226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8ad6b-a7ec-4adb-bacf-ceb2ecf63e35 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.096683 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8ad6b-a7ec-4adb-bacf-ceb2ecf63e35 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.101428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2984bde-dfc8-4e8d-9c58-64bf8f2b611e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.102099 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2984bde-dfc8-4e8d-9c58-64bf8f2b611e admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.104209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2984bde-dfc8-4e8d-9c58-64bf8f2b611e admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.104209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2984bde-dfc8-4e8d-9c58-64bf8f2b611e admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.104209 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2984bde-dfc8-4e8d-9c58-64bf8f2b611e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.105575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2984bde-dfc8-4e8d-9c58-64bf8f2b611e admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.116231 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1351/2661] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:04:00.120483 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2984bde-dfc8-4e8d-9c58-64bf8f2b611e admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.130135 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2984bde-dfc8-4e8d-9c58-64bf8f2b611e admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.137024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9f79c5af-f517-4a51-a4f5-6eb73115d72c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bf10619183043fbbd991e3dad00c4a8', 'request_id': 'req-9f79c5af-f517-4a51-a4f5-6eb73115d72c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:00.143726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9f79c5af-f517-4a51-a4f5-6eb73115d72c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bf10619183043fbbd991e3dad00c4a8', 'request_id': 'req-9f79c5af-f517-4a51-a4f5-6eb73115d72c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:00.185410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9f79c5af-f517-4a51-a4f5-6eb73115d72c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bf10619183043fbbd991e3dad00c4a8', 'request_id': 'req-9f79c5af-f517-4a51-a4f5-6eb73115d72c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:00.188360 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de92cc0-3449-4974-9d0e-13c9028c18db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.189016 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de92cc0-3449-4974-9d0e-13c9028c18db admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.189299 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de92cc0-3449-4974-9d0e-13c9028c18db admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.189569 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de92cc0-3449-4974-9d0e-13c9028c18db admin admin] PATH_INFO: `/v3/projects/e9fa4136493f4bb89ffe5381a6415f71/users/2040bd95536d4244a6fe56c91ac40428/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.197552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9f79c5af-f517-4a51-a4f5-6eb73115d72c admin admin] The token cache is being invalidate because project 4bf10619183043fbbd991e3dad00c4a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:04:00.199367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9f79c5af-f517-4a51-a4f5-6eb73115d72c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:00.199367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-9f79c5af-f517-4a51-a4f5-6eb73115d72c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:00.200090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1311/2662] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:04:00 2022] DELETE /identity/v3/projects/4bf10619183043fbbd991e3dad00c4a8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:04:00.205435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de92cc0-3449-4974-9d0e-13c9028c18db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9fa4136493f4bb89ffe5381a6415f71, user_id=2040bd95536d4244a6fe56c91ac40428, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.205849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45a762-3f80-444a-a730-69869b9ae591 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.206136 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45a762-3f80-444a-a730-69869b9ae591 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.206515 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45a762-3f80-444a-a730-69869b9ae591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.208502 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de92cc0-3449-4974-9d0e-13c9028c18db admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.228118 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-8d45a762-3f80-444a-a730-69869b9ae591 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:00.229871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-8d45a762-3f80-444a-a730-69869b9ae591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:00.238978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1312/2663] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:04:00.247843 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc037cc-0a03-4241-9b4f-df8ab546eb93 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.248124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc037cc-0a03-4241-9b4f-df8ab546eb93 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.248393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc037cc-0a03-4241-9b4f-df8ab546eb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.283840 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1352/2664] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:04:00 2022] PUT /identity/v3/projects/e9fa4136493f4bb89ffe5381a6415f71/users/2040bd95536d4244a6fe56c91ac40428/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:04:00.295651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451b063-b9d7-4198-851a-9a9ab7f12df9 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.295937 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451b063-b9d7-4198-851a-9a9ab7f12df9 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.296212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451b063-b9d7-4198-851a-9a9ab7f12df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.328785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-a451b063-b9d7-4198-851a-9a9ab7f12df9 None None] MFA Rules not processed for user `2040bd95536d4244a6fe56c91ac40428`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:00.330026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a451b063-b9d7-4198-851a-9a9ab7f12df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:00.375894 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:00.388247 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-0dc037cc-0a03-4241-9b4f-df8ab546eb93 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:00.389448 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0dc037cc-0a03-4241-9b4f-df8ab546eb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:00.400755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1353/2665] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:04:00.405266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451b063-b9d7-4198-851a-9a9ab7f12df9 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.406636 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a451b063-b9d7-4198-851a-9a9ab7f12df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:00.409227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:00.449101 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1313/2666] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:04:00.453347 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc037cc-0a03-4241-9b4f-df8ab546eb93 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.463938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc037cc-0a03-4241-9b4f-df8ab546eb93 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.468499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0ebf7-4bb6-4446-8cf1-520a47d9df8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.469076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0ebf7-4bb6-4446-8cf1-520a47d9df8d admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.469350 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0ebf7-4bb6-4446-8cf1-520a47d9df8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.469615 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0ebf7-4bb6-4446-8cf1-520a47d9df8d admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.470241 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0ebf7-4bb6-4446-8cf1-520a47d9df8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:04:00.470571 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0ebf7-4bb6-4446-8cf1-520a47d9df8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.472141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0ebf7-4bb6-4446-8cf1-520a47d9df8d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.472669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451b063-b9d7-4198-851a-9a9ab7f12df9 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.477429 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443b575-891e-45a2-b63a-59ff917ca16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.478022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443b575-891e-45a2-b63a-59ff917ca16d admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.478293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443b575-891e-45a2-b63a-59ff917ca16d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.478592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443b575-891e-45a2-b63a-59ff917ca16d admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.479214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443b575-891e-45a2-b63a-59ff917ca16d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.481048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443b575-891e-45a2-b63a-59ff917ca16d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.483775 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1314/2667] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:04:00.488124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0ebf7-4bb6-4446-8cf1-520a47d9df8d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.498613 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0ebf7-4bb6-4446-8cf1-520a47d9df8d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.504444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1aed6-0329-49cf-a228-46ee6f78e8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.504444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1aed6-0329-49cf-a228-46ee6f78e8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.504444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1aed6-0329-49cf-a228-46ee6f78e8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.504444 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1aed6-0329-49cf-a228-46ee6f78e8b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.504937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1aed6-0329-49cf-a228-46ee6f78e8b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.506879 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1aed6-0329-49cf-a228-46ee6f78e8b6 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.581190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1354/2668] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:04:00 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:04:00.585288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443b575-891e-45a2-b63a-59ff917ca16d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.605623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1315/2669] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:04:00 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:04:00.611237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1aed6-0329-49cf-a228-46ee6f78e8b6 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.652671 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443b575-891e-45a2-b63a-59ff917ca16d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.674642 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1aed6-0329-49cf-a228-46ee6f78e8b6 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.680307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6742cc-5acc-459e-9c6d-4b6bc5bc0cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.681124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6742cc-5acc-459e-9c6d-4b6bc5bc0cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.681568 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6742cc-5acc-459e-9c6d-4b6bc5bc0cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.681865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6742cc-5acc-459e-9c6d-4b6bc5bc0cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.682620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6742cc-5acc-459e-9c6d-4b6bc5bc0cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.684507 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6742cc-5acc-459e-9c6d-4b6bc5bc0cd5 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.708838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e2bb0-736d-46d3-a364-be27f892a93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.709467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e2bb0-736d-46d3-a364-be27f892a93f admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.709713 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e2bb0-736d-46d3-a364-be27f892a93f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.709980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e2bb0-736d-46d3-a364-be27f892a93f admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.710635 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e2bb0-736d-46d3-a364-be27f892a93f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.712675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e2bb0-736d-46d3-a364-be27f892a93f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.768617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1316/2670] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:04:00 2022] POST /identity/v3/users => generated 291 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:04:00.772540 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6742cc-5acc-459e-9c6d-4b6bc5bc0cd5 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.781961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6742cc-5acc-459e-9c6d-4b6bc5bc0cd5 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.786698 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca65c0-7ce8-4e04-b8f6-5e15bb132a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.787279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca65c0-7ce8-4e04-b8f6-5e15bb132a0a admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.787589 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca65c0-7ce8-4e04-b8f6-5e15bb132a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.787864 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca65c0-7ce8-4e04-b8f6-5e15bb132a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.788494 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca65c0-7ce8-4e04-b8f6-5e15bb132a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.790783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca65c0-7ce8-4e04-b8f6-5e15bb132a0a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.792568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1355/2671] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:04:00 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:04:00.796922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e2bb0-736d-46d3-a364-be27f892a93f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.801541 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1317/2672] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:04:00.801846 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:00.805816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca65c0-7ce8-4e04-b8f6-5e15bb132a0a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.807838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e2bb0-736d-46d3-a364-be27f892a93f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.812437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57756c2a-6362-473a-b4bc-eb495dc66b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.813026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57756c2a-6362-473a-b4bc-eb495dc66b3a admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.813335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57756c2a-6362-473a-b4bc-eb495dc66b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.813577 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57756c2a-6362-473a-b4bc-eb495dc66b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.814288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57756c2a-6362-473a-b4bc-eb495dc66b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.816133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57756c2a-6362-473a-b4bc-eb495dc66b3a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.816232 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca65c0-7ce8-4e04-b8f6-5e15bb132a0a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.821144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe89344-d10f-4b09-bc64-a966f990fd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.821833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe89344-d10f-4b09-bc64-a966f990fd90 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.822105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe89344-d10f-4b09-bc64-a966f990fd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.822510 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe89344-d10f-4b09-bc64-a966f990fd90 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.823173 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe89344-d10f-4b09-bc64-a966f990fd90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.824921 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe89344-d10f-4b09-bc64-a966f990fd90 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.826706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1356/2673] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:04:00.836913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57756c2a-6362-473a-b4bc-eb495dc66b3a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.840005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1318/2674] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:04:00.840984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe89344-d10f-4b09-bc64-a966f990fd90 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.848515 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57756c2a-6362-473a-b4bc-eb495dc66b3a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.850813 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe89344-d10f-4b09-bc64-a966f990fd90 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:00.853830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8209be9-a4fb-49aa-80c9-2ade4a388f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.854519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8209be9-a4fb-49aa-80c9-2ade4a388f6c admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.854798 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8209be9-a4fb-49aa-80c9-2ade4a388f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.855067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8209be9-a4fb-49aa-80c9-2ade4a388f6c admin admin] PATH_INFO: `/v3/projects/7d9ac6fac1e548ad898e1123baea8424/users/1c72c141595e4970a6fcce1f0876ad0e/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.856740 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4fc19-ad26-4576-9d84-66fc289887d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:00.856878 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4fc19-ad26-4576-9d84-66fc289887d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.857019 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4fc19-ad26-4576-9d84-66fc289887d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.857300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4fc19-ad26-4576-9d84-66fc289887d9 admin admin] PATH_INFO: `/v3/projects/5c666d7da84e406ea29cd81fc22bb1bd/users/3902c87ad7d14763ae7f4924d0d4b22c/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:00.871428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8209be9-a4fb-49aa-80c9-2ade4a388f6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d9ac6fac1e548ad898e1123baea8424, user_id=1c72c141595e4970a6fcce1f0876ad0e, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.872461 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4fc19-ad26-4576-9d84-66fc289887d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c666d7da84e406ea29cd81fc22bb1bd, user_id=3902c87ad7d14763ae7f4924d0d4b22c, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:00.874067 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8209be9-a4fb-49aa-80c9-2ade4a388f6c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.875122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4fc19-ad26-4576-9d84-66fc289887d9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:00.951738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1357/2675] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:04:00 2022] PUT /identity/v3/projects/7d9ac6fac1e548ad898e1123baea8424/users/1c72c141595e4970a6fcce1f0876ad0e/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:04:00.955702 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8209be9-a4fb-49aa-80c9-2ade4a388f6c admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:00.986037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1319/2676] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:04:00 2022] PUT /identity/v3/projects/5c666d7da84e406ea29cd81fc22bb1bd/users/3902c87ad7d14763ae7f4924d0d4b22c/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:04:00.997623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a68e0-df33-4943-af53-a7d47417b8b2 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:00.997970 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a68e0-df33-4943-af53-a7d47417b8b2 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:00.998306 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a68e0-df33-4943-af53-a7d47417b8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:01.018210 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8209be9-a4fb-49aa-80c9-2ade4a388f6c admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:01.028171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-675a68e0-df33-4943-af53-a7d47417b8b2 None None] MFA Rules not processed for user `3902c87ad7d14763ae7f4924d0d4b22c`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:01.029383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-675a68e0-df33-4943-af53-a7d47417b8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:01.049886 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:01.076860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fb850-ad25-4464-9314-5bf2132e2b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:01.077464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fb850-ad25-4464-9314-5bf2132e2b1b admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:01.077741 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fb850-ad25-4464-9314-5bf2132e2b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:01.078009 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fb850-ad25-4464-9314-5bf2132e2b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:01.078678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fb850-ad25-4464-9314-5bf2132e2b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:01.080246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fb850-ad25-4464-9314-5bf2132e2b1b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:01.092459 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1358/2677] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:04:01.094915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fb850-ad25-4464-9314-5bf2132e2b1b admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:01.096309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1320/2678] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:04:01.108771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a68e0-df33-4943-af53-a7d47417b8b2 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:01.109134 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fb850-ad25-4464-9314-5bf2132e2b1b admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:01.110283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-675a68e0-df33-4943-af53-a7d47417b8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:01.114225 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905597ea-dd64-4da3-9c48-03730a73371d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:01.114917 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905597ea-dd64-4da3-9c48-03730a73371d admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:01.115481 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905597ea-dd64-4da3-9c48-03730a73371d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:01.115778 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905597ea-dd64-4da3-9c48-03730a73371d admin admin] PATH_INFO: `/v3/projects/7d9ac6fac1e548ad898e1123baea8424/users/1c72c141595e4970a6fcce1f0876ad0e/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:01.117884 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905597ea-dd64-4da3-9c48-03730a73371d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d9ac6fac1e548ad898e1123baea8424, user_id=1c72c141595e4970a6fcce1f0876ad0e, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:01.120611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905597ea-dd64-4da3-9c48-03730a73371d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:01.131464 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1359/2679] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:04:01 2022] PUT /identity/v3/projects/7d9ac6fac1e548ad898e1123baea8424/users/1c72c141595e4970a6fcce1f0876ad0e/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:04:01.142540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dd4dc-a5c4-45c0-babc-db9fc438ee06 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:01.142827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dd4dc-a5c4-45c0-babc-db9fc438ee06 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:01.143102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dd4dc-a5c4-45c0-babc-db9fc438ee06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:01.167892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a68e0-df33-4943-af53-a7d47417b8b2 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:01.172120 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-6e0dd4dc-a5c4-45c0-babc-db9fc438ee06 None None] MFA Rules not processed for user `1c72c141595e4970a6fcce1f0876ad0e`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:01.173513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6e0dd4dc-a5c4-45c0-babc-db9fc438ee06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:01.216932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b49b3-009b-4b5b-9e96-59559554a940 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:01.217540 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b49b3-009b-4b5b-9e96-59559554a940 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:01.217820 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b49b3-009b-4b5b-9e96-59559554a940 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:01.218091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b49b3-009b-4b5b-9e96-59559554a940 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:01.219320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:01.242835 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1360/2680] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:04:01.255641 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dd4dc-a5c4-45c0-babc-db9fc438ee06 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:01.260592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6e0dd4dc-a5c4-45c0-babc-db9fc438ee06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:01.271538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dd4dc-a5c4-45c0-babc-db9fc438ee06 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:01.276978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48515e-aa28-489f-a6d9-f5a06f4f0d21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:01.277779 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48515e-aa28-489f-a6d9-f5a06f4f0d21 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:01.283404 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48515e-aa28-489f-a6d9-f5a06f4f0d21 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:01.283404 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48515e-aa28-489f-a6d9-f5a06f4f0d21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:01.283619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b49b3-009b-4b5b-9e96-59559554a940 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:01.283619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b49b3-009b-4b5b-9e96-59559554a940 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:01.288236 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48515e-aa28-489f-a6d9-f5a06f4f0d21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:01.290451 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48515e-aa28-489f-a6d9-f5a06f4f0d21 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:01.292491 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1321/2681] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:04:01.302233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1361/2682] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:04:01.312571 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b49b3-009b-4b5b-9e96-59559554a940 service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:01.348733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:01.376866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b49b3-009b-4b5b-9e96-59559554a940 service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:01.381912 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:01.382561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:01.382842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:01.383694 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] PATH_INFO: `/v3/users/3902c87ad7d14763ae7f4924d0d4b22c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:01.385169 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3902c87ad7d14763ae7f4924d0d4b22c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:01.387098 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:01.465642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48515e-aa28-489f-a6d9-f5a06f4f0d21 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:01.474379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48515e-aa28-489f-a6d9-f5a06f4f0d21 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:01.478253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da29ed-8ac2-4941-bb32-9aab05225d7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:01.479968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da29ed-8ac2-4941-bb32-9aab05225d7f service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:01.479968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da29ed-8ac2-4941-bb32-9aab05225d7f service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:01.479968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da29ed-8ac2-4941-bb32-9aab05225d7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:01.535860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da29ed-8ac2-4941-bb32-9aab05225d7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:01.537294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da29ed-8ac2-4941-bb32-9aab05225d7f service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:01.546190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1362/2683] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:04:01.573226 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3902c87ad7d14763ae7f4924d0d4b22c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:01.642769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3902c87ad7d14763ae7f4924d0d4b22c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:01.643336 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3902c87ad7d14763ae7f4924d0d4b22c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:01.643972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3902c87ad7d14763ae7f4924d0d4b22c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:01.652138 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3902c87ad7d14763ae7f4924d0d4b22c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:01.669777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1322/2684] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:04:01 2022] DELETE /identity/v3/users/3902c87ad7d14763ae7f4924d0d4b22c => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:01.675132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773e88d-0b46-4752-869c-b3d0f3f00f6a None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:01.675601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773e88d-0b46-4752-869c-b3d0f3f00f6a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:01.675777 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773e88d-0b46-4752-869c-b3d0f3f00f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:01.694496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-6773e88d-0b46-4752-869c-b3d0f3f00f6a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:01.696199 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6773e88d-0b46-4752-869c-b3d0f3f00f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:01.761927 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1363/2685] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:01.772228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:01.774730 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:01.833986 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f90de7-b9d6-4ec4-b02a-14701950d99f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:01.837674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293066dc-076f-40ba-88e9-eec0362b47ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:01.838303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293066dc-076f-40ba-88e9-eec0362b47ad service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:01.838611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293066dc-076f-40ba-88e9-eec0362b47ad service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:01.838887 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293066dc-076f-40ba-88e9-eec0362b47ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:01.847815 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293066dc-076f-40ba-88e9-eec0362b47ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:01.849264 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293066dc-076f-40ba-88e9-eec0362b47ad service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:01.858883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1323/2686] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:04:02.028626 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773e88d-0b46-4752-869c-b3d0f3f00f6a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:02.030061 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6773e88d-0b46-4752-869c-b3d0f3f00f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:02.041537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773e88d-0b46-4752-869c-b3d0f3f00f6a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:02.046229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d325d8-dd57-411e-ab1c-df72e13f74b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:02.047425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d325d8-dd57-411e-ab1c-df72e13f74b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:02.047425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d325d8-dd57-411e-ab1c-df72e13f74b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:02.047704 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d325d8-dd57-411e-ab1c-df72e13f74b6 admin admin] PATH_INFO: `/v3/users/2040bd95536d4244a6fe56c91ac40428` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:02.048567 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d325d8-dd57-411e-ab1c-df72e13f74b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2040bd95536d4244a6fe56c91ac40428)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:02.050449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d325d8-dd57-411e-ab1c-df72e13f74b6 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:02.182588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-64d325d8-dd57-411e-ab1c-df72e13f74b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2040bd95536d4244a6fe56c91ac40428'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:02.231277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-64d325d8-dd57-411e-ab1c-df72e13f74b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2040bd95536d4244a6fe56c91ac40428'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:02.231890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-64d325d8-dd57-411e-ab1c-df72e13f74b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2040bd95536d4244a6fe56c91ac40428'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:02.232494 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-64d325d8-dd57-411e-ab1c-df72e13f74b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2040bd95536d4244a6fe56c91ac40428'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:02.240159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:02.245723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-64d325d8-dd57-411e-ab1c-df72e13f74b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2040bd95536d4244a6fe56c91ac40428'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:02.261818 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1364/2687] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:04:02 2022] DELETE /identity/v3/users/2040bd95536d4244a6fe56c91ac40428 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:02.263074 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293066dc-076f-40ba-88e9-eec0362b47ad service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:02.263543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5d37d-ece1-4f06-8c77-d4c87863b0fd None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:02.263543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5d37d-ece1-4f06-8c77-d4c87863b0fd None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:02.264033 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5d37d-ece1-4f06-8c77-d4c87863b0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:02.264520 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-293066dc-076f-40ba-88e9-eec0362b47ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:02.280277 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293066dc-076f-40ba-88e9-eec0362b47ad service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:02.298093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-f9c5d37d-ece1-4f06-8c77-d4c87863b0fd None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:02.299764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f9c5d37d-ece1-4f06-8c77-d4c87863b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:02.337986 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:02.344443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85b5ff-9613-4380-a757-e724b9ccb7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:02.345098 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85b5ff-9613-4380-a757-e724b9ccb7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:02.345380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85b5ff-9613-4380-a757-e724b9ccb7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:02.345647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85b5ff-9613-4380-a757-e724b9ccb7c1 admin admin] PATH_INFO: `/v3/projects/5c666d7da84e406ea29cd81fc22bb1bd` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:02.346822 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85b5ff-9613-4380-a757-e724b9ccb7c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c666d7da84e406ea29cd81fc22bb1bd)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:02.348726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85b5ff-9613-4380-a757-e724b9ccb7c1 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:02.367108 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1365/2688] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:04:02.377989 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5d37d-ece1-4f06-8c77-d4c87863b0fd None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:02.379545 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f9c5d37d-ece1-4f06-8c77-d4c87863b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:02.426657 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3b85b5ff-9613-4380-a757-e724b9ccb7c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c666d7da84e406ea29cd81fc22bb1bd', 'request_id': 'req-3b85b5ff-9613-4380-a757-e724b9ccb7c1'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:02.432933 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3b85b5ff-9613-4380-a757-e724b9ccb7c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c666d7da84e406ea29cd81fc22bb1bd', 'request_id': 'req-3b85b5ff-9613-4380-a757-e724b9ccb7c1'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:02.440391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5d37d-ece1-4f06-8c77-d4c87863b0fd None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:02.487751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64861bb-b0e8-4fad-9aba-3e9e417e0ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:02.488342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64861bb-b0e8-4fad-9aba-3e9e417e0ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:02.488617 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64861bb-b0e8-4fad-9aba-3e9e417e0ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:02.488882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64861bb-b0e8-4fad-9aba-3e9e417e0ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:02.499793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3b85b5ff-9613-4380-a757-e724b9ccb7c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c666d7da84e406ea29cd81fc22bb1bd', 'request_id': 'req-3b85b5ff-9613-4380-a757-e724b9ccb7c1'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:02.512038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3b85b5ff-9613-4380-a757-e724b9ccb7c1 admin admin] The token cache is being invalidate because project 5c666d7da84e406ea29cd81fc22bb1bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:04:02.512336 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3b85b5ff-9613-4380-a757-e724b9ccb7c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:02.512984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-3b85b5ff-9613-4380-a757-e724b9ccb7c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:02.514561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1324/2689] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:04:02 2022] DELETE /identity/v3/projects/5c666d7da84e406ea29cd81fc22bb1bd => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:02.519234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23df34-b1bc-40dd-919c-c1c3f73f0387 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:02.519528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23df34-b1bc-40dd-919c-c1c3f73f0387 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:02.521726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23df34-b1bc-40dd-919c-c1c3f73f0387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:02.540689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:02.541604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-2d23df34-b1bc-40dd-919c-c1c3f73f0387 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:02.543000 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2d23df34-b1bc-40dd-919c-c1c3f73f0387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:02.553711 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64861bb-b0e8-4fad-9aba-3e9e417e0ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:02.555886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64861bb-b0e8-4fad-9aba-3e9e417e0ad7 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:02.565365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1366/2690] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:04:02.608617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1325/2691] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:04:02.616322 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea791-46d8-4484-926a-6a82a4f58b7f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:02.616603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea791-46d8-4484-926a-6a82a4f58b7f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:02.616873 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea791-46d8-4484-926a-6a82a4f58b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:02.636916 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-6d9ea791-46d8-4484-926a-6a82a4f58b7f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:02.638102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6d9ea791-46d8-4484-926a-6a82a4f58b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:02.645950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1367/2692] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:02.651191 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23df34-b1bc-40dd-919c-c1c3f73f0387 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:02.660996 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23df34-b1bc-40dd-919c-c1c3f73f0387 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:02.667039 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ef9bf-c7eb-4b3d-b6be-52d94f237c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:02.667535 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ef9bf-c7eb-4b3d-b6be-52d94f237c30 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:02.667843 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ef9bf-c7eb-4b3d-b6be-52d94f237c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:02.668112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ef9bf-c7eb-4b3d-b6be-52d94f237c30 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:02.668752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ef9bf-c7eb-4b3d-b6be-52d94f237c30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:04:02.669051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ef9bf-c7eb-4b3d-b6be-52d94f237c30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:02.670650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ef9bf-c7eb-4b3d-b6be-52d94f237c30 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:02.682929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1326/2693] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:04:02.687200 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ea791-46d8-4484-926a-6a82a4f58b7f None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:02.696648 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ea791-46d8-4484-926a-6a82a4f58b7f None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:02.701215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52a2a8-871a-4be8-9ca1-5884d39284ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:02.701797 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52a2a8-871a-4be8-9ca1-5884d39284ac admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:02.702070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52a2a8-871a-4be8-9ca1-5884d39284ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:02.702336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52a2a8-871a-4be8-9ca1-5884d39284ac admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:02.703026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52a2a8-871a-4be8-9ca1-5884d39284ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:02.704880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52a2a8-871a-4be8-9ca1-5884d39284ac admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:02.815755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1368/2694] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:04:02 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:04:02.819915 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ef9bf-c7eb-4b3d-b6be-52d94f237c30 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:02.830297 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ef9bf-c7eb-4b3d-b6be-52d94f237c30 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:02.888315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e310c13-373b-4c66-9f89-c401e42f90c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:02.888923 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e310c13-373b-4c66-9f89-c401e42f90c5 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:02.889204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e310c13-373b-4c66-9f89-c401e42f90c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:02.889477 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e310c13-373b-4c66-9f89-c401e42f90c5 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:02.890102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e310c13-373b-4c66-9f89-c401e42f90c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:02.892038 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e310c13-373b-4c66-9f89-c401e42f90c5 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:02.991534 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1327/2695] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:04:02 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:04:02.996644 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e310c13-373b-4c66-9f89-c401e42f90c5 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:02.996858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52a2a8-871a-4be8-9ca1-5884d39284ac admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:03.003613 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1a52a2a8-871a-4be8-9ca1-5884d39284ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:03.003946 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:03.012378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e310c13-373b-4c66-9f89-c401e42f90c5 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:03.012961 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52a2a8-871a-4be8-9ca1-5884d39284ac admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:03.018976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31de4a-654b-43e3-b55c-af414d14b545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:03.019171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:03.019617 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31de4a-654b-43e3-b55c-af414d14b545 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:03.019872 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31de4a-654b-43e3-b55c-af414d14b545 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:03.020144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31de4a-654b-43e3-b55c-af414d14b545 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:03.020787 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31de4a-654b-43e3-b55c-af414d14b545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:03.020902 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:03.021662 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:03.022295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] PATH_INFO: `/v3/projects/e9fa4136493f4bb89ffe5381a6415f71` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:03.022391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31de4a-654b-43e3-b55c-af414d14b545 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:03.024909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9fa4136493f4bb89ffe5381a6415f71)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:03.028379 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:03.034810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1328/2696] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:04:03.038637 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31de4a-654b-43e3-b55c-af414d14b545 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:03.050033 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31de4a-654b-43e3-b55c-af414d14b545 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:03.055892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c7a78-62f3-458e-a341-146e065440a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:03.060985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c7a78-62f3-458e-a341-146e065440a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:03.060985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c7a78-62f3-458e-a341-146e065440a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:03.060985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c7a78-62f3-458e-a341-146e065440a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:03.060985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c7a78-62f3-458e-a341-146e065440a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:03.060985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c7a78-62f3-458e-a341-146e065440a8 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:03.069938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1329/2697] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:04:03.074008 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c7a78-62f3-458e-a341-146e065440a8 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:03.084792 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c7a78-62f3-458e-a341-146e065440a8 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:03.089762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9295c8-d3f8-4709-85b6-7392943c36c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:03.090391 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9295c8-d3f8-4709-85b6-7392943c36c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:03.090704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9295c8-d3f8-4709-85b6-7392943c36c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:03.090998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9295c8-d3f8-4709-85b6-7392943c36c9 admin admin] PATH_INFO: `/v3/projects/c5d9325889ba4e55904ddea35f68dd33/users/cd7fdf9c3d0c444e987e2a0873b6602b/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:03.106029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9295c8-d3f8-4709-85b6-7392943c36c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5d9325889ba4e55904ddea35f68dd33, user_id=cd7fdf9c3d0c444e987e2a0873b6602b, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:03.108659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9295c8-d3f8-4709-85b6-7392943c36c9 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:03.197026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1330/2698] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:04:03 2022] PUT /identity/v3/projects/c5d9325889ba4e55904ddea35f68dd33/users/cd7fdf9c3d0c444e987e2a0873b6602b/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:04:03.204331 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9fa4136493f4bb89ffe5381a6415f71', 'request_id': 'req-6e30db84-5246-455b-9120-3b09507b3fe6'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:03.208470 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46bdfb-cf72-47e4-9f36-bd7be6d1901e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:03.208753 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46bdfb-cf72-47e4-9f36-bd7be6d1901e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:03.209024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46bdfb-cf72-47e4-9f36-bd7be6d1901e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:03.211074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9fa4136493f4bb89ffe5381a6415f71', 'request_id': 'req-6e30db84-5246-455b-9120-3b09507b3fe6'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:03.237207 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-5d46bdfb-cf72-47e4-9f36-bd7be6d1901e None None] MFA Rules not processed for user `cd7fdf9c3d0c444e987e2a0873b6602b`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:03.238327 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5d46bdfb-cf72-47e4-9f36-bd7be6d1901e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:03.276670 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9fa4136493f4bb89ffe5381a6415f71', 'request_id': 'req-6e30db84-5246-455b-9120-3b09507b3fe6'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:03.287913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] The token cache is being invalidate because project e9fa4136493f4bb89ffe5381a6415f71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:04:03.288215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:03.288881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:03.293219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1369/2699] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:04:02 2022] DELETE /identity/v3/projects/e9fa4136493f4bb89ffe5381a6415f71 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:03.294789 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:03.296179 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:03.307783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1331/2700] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:04:03.312289 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46bdfb-cf72-47e4-9f36-bd7be6d1901e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:03.313594 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5d46bdfb-cf72-47e4-9f36-bd7be6d1901e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:03.364587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30db84-5246-455b-9120-3b09507b3fe6 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:03.370222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:03.370881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:03.371165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:03.371438 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] PATH_INFO: `/v3/users/1c72c141595e4970a6fcce1f0876ad0e` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:03.372585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c72c141595e4970a6fcce1f0876ad0e)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:03.374526 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:03.379897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46bdfb-cf72-47e4-9f36-bd7be6d1901e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:03.385156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8af813-ef9e-44f3-a927-afcd15a789fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:03.385763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8af813-ef9e-44f3-a927-afcd15a789fb admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:03.386042 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8af813-ef9e-44f3-a927-afcd15a789fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:03.386311 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8af813-ef9e-44f3-a927-afcd15a789fb admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:03.386967 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8af813-ef9e-44f3-a927-afcd15a789fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:03.388869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8af813-ef9e-44f3-a927-afcd15a789fb admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:03.472894 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1332/2701] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:04:03 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:04:03.477317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8af813-ef9e-44f3-a927-afcd15a789fb admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:03.511218 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:03.539067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8af813-ef9e-44f3-a927-afcd15a789fb admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:03.540201 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:03.542892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c72c141595e4970a6fcce1f0876ad0e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:03.544833 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6acb7f-078a-4f71-822f-5e276fef7f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:03.545671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6acb7f-078a-4f71-822f-5e276fef7f68 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:03.546379 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6acb7f-078a-4f71-822f-5e276fef7f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:03.546721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6acb7f-078a-4f71-822f-5e276fef7f68 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:03.547351 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6acb7f-078a-4f71-822f-5e276fef7f68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:03.549231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6acb7f-078a-4f71-822f-5e276fef7f68 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:03.579509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c72c141595e4970a6fcce1f0876ad0e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:03.580219 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c72c141595e4970a6fcce1f0876ad0e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:03.580849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c72c141595e4970a6fcce1f0876ad0e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:03.593485 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c72c141595e4970a6fcce1f0876ad0e'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:03.606426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1370/2702] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:04:03 2022] DELETE /identity/v3/users/1c72c141595e4970a6fcce1f0876ad0e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:03.621059 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1333/2703] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:04:03 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:04:03.625152 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:03.627206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:03.694327 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb164b83-87d1-423f-8881-43e886aaa620 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:03.699476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b469f-bbf6-4cb6-9680-da2d7eea4017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:03.700406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b469f-bbf6-4cb6-9680-da2d7eea4017 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:03.700690 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b469f-bbf6-4cb6-9680-da2d7eea4017 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:03.700960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b469f-bbf6-4cb6-9680-da2d7eea4017 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:03.701608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b469f-bbf6-4cb6-9680-da2d7eea4017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:03.703261 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b469f-bbf6-4cb6-9680-da2d7eea4017 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:03.713967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1371/2704] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:04:03.717989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6acb7f-078a-4f71-822f-5e276fef7f68 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:03.728907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6acb7f-078a-4f71-822f-5e276fef7f68 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:03.733528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dda2c7-6145-4e4d-8a9a-0a3bd55b452c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:03.734112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dda2c7-6145-4e4d-8a9a-0a3bd55b452c admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:03.734387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dda2c7-6145-4e4d-8a9a-0a3bd55b452c admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:03.734691 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dda2c7-6145-4e4d-8a9a-0a3bd55b452c admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:03.735317 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dda2c7-6145-4e4d-8a9a-0a3bd55b452c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:03.736924 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dda2c7-6145-4e4d-8a9a-0a3bd55b452c admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:03.750580 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1334/2705] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:04:03.751986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b469f-bbf6-4cb6-9680-da2d7eea4017 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:03.762663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b469f-bbf6-4cb6-9680-da2d7eea4017 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:03.767413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5214243-e179-460b-a476-01eb4751e004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:03.768062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5214243-e179-460b-a476-01eb4751e004 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:03.768343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5214243-e179-460b-a476-01eb4751e004 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:03.768618 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5214243-e179-460b-a476-01eb4751e004 admin admin] PATH_INFO: `/v3/projects/766860b5768240e992174761f07b4142/users/30cd98d792b24217b1efa3934d1fd081/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:03.785892 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5214243-e179-460b-a476-01eb4751e004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=766860b5768240e992174761f07b4142, user_id=30cd98d792b24217b1efa3934d1fd081, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:03.788676 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5214243-e179-460b-a476-01eb4751e004 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:03.876562 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1372/2706] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:04:03 2022] PUT /identity/v3/projects/766860b5768240e992174761f07b4142/users/30cd98d792b24217b1efa3934d1fd081/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:03.887260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38372a4d-155c-4b3b-8a86-cc98744b5e4e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:03.887653 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38372a4d-155c-4b3b-8a86-cc98744b5e4e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:03.887934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38372a4d-155c-4b3b-8a86-cc98744b5e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:03.919894 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-38372a4d-155c-4b3b-8a86-cc98744b5e4e None None] MFA Rules not processed for user `30cd98d792b24217b1efa3934d1fd081`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:03.921110 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-38372a4d-155c-4b3b-8a86-cc98744b5e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:03.982406 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5214243-e179-460b-a476-01eb4751e004 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:03.984519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a5214243-e179-460b-a476-01eb4751e004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:03.986931 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1335/2707] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:04:03.991145 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38372a4d-155c-4b3b-8a86-cc98744b5e4e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:04.052195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5214243-e179-460b-a476-01eb4751e004 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.057252 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.057920 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.058216 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.058584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8 admin admin] PATH_INFO: `/v3/projects/7d9ac6fac1e548ad898e1123baea8424` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.059718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d9ac6fac1e548ad898e1123baea8424)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.060262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38372a4d-155c-4b3b-8a86-cc98744b5e4e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.061638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.064917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17879c-9d06-465e-bfdd-c84f04ce2df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.065548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17879c-9d06-465e-bfdd-c84f04ce2df7 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.065826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17879c-9d06-465e-bfdd-c84f04ce2df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.066096 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17879c-9d06-465e-bfdd-c84f04ce2df7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.066751 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17879c-9d06-465e-bfdd-c84f04ce2df7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.068621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17879c-9d06-465e-bfdd-c84f04ce2df7 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.164307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1336/2708] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:04:03 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:04:04.168959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17879c-9d06-465e-bfdd-c84f04ce2df7 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:04.176665 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:04.187725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d9ac6fac1e548ad898e1123baea8424', 'request_id': 'req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:04.194134 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d9ac6fac1e548ad898e1123baea8424', 'request_id': 'req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:04.197024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:04.231567 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17879c-9d06-465e-bfdd-c84f04ce2df7 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.242062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d9ac6fac1e548ad898e1123baea8424', 'request_id': 'req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:04.253563 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8 admin admin] The token cache is being invalidate because project 7d9ac6fac1e548ad898e1123baea8424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:04:04.254294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:04.254849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-16ebcc45-c493-4d62-ba69-f8a36a9d6ae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:04.256414 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1373/2709] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:04:03 2022] DELETE /identity/v3/projects/7d9ac6fac1e548ad898e1123baea8424 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:04:04.261616 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130590fd-4ad7-42df-810c-0c8d21409ce7 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.261898 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130590fd-4ad7-42df-810c-0c8d21409ce7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.262171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130590fd-4ad7-42df-810c-0c8d21409ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.283104 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-130590fd-4ad7-42df-810c-0c8d21409ce7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:04.284827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-130590fd-4ad7-42df-810c-0c8d21409ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:04.288430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2d15f-e424-4152-a403-0939a9cfd158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.289180 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2d15f-e424-4152-a403-0939a9cfd158 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.289762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2d15f-e424-4152-a403-0939a9cfd158 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.290207 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2d15f-e424-4152-a403-0939a9cfd158 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.290877 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2d15f-e424-4152-a403-0939a9cfd158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.293016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2d15f-e424-4152-a403-0939a9cfd158 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.293546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1374/2710] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:04:04.301148 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40a855-c17b-4123-94c9-1b197290e2f0 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.307360 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40a855-c17b-4123-94c9-1b197290e2f0 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.307360 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40a855-c17b-4123-94c9-1b197290e2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.323137 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0c40a855-c17b-4123-94c9-1b197290e2f0 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:04.324334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0c40a855-c17b-4123-94c9-1b197290e2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:04.332399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1375/2711] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:04:04.337540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40a855-c17b-4123-94c9-1b197290e2f0 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:04.346513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40a855-c17b-4123-94c9-1b197290e2f0 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.351161 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d172f-9f19-4789-91fe-33d5825c44f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.351725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d172f-9f19-4789-91fe-33d5825c44f6 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.352762 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d172f-9f19-4789-91fe-33d5825c44f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.353040 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d172f-9f19-4789-91fe-33d5825c44f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.353672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d172f-9f19-4789-91fe-33d5825c44f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:04:04.353966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d172f-9f19-4789-91fe-33d5825c44f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.355544 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d172f-9f19-4789-91fe-33d5825c44f6 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.367085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1376/2712] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 13:04:04.371096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d172f-9f19-4789-91fe-33d5825c44f6 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:04.383776 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1337/2713] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:04:04 2022] POST /identity/v3/users => generated 295 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:04:04.384276 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d172f-9f19-4789-91fe-33d5825c44f6 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.384964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2d15f-e424-4152-a403-0939a9cfd158 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:04.386585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a9b2d15f-e424-4152-a403-0939a9cfd158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:04.386702 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cc14f-70a6-4f21-8ccb-11c4823d640a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.391870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cc14f-70a6-4f21-8ccb-11c4823d640a admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.391870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cc14f-70a6-4f21-8ccb-11c4823d640a admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.391870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cc14f-70a6-4f21-8ccb-11c4823d640a admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.391870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0cc14f-70a6-4f21-8ccb-11c4823d640a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.391870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0cc14f-70a6-4f21-8ccb-11c4823d640a admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.401124 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2d15f-e424-4152-a403-0939a9cfd158 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.407744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80029da6-415f-4239-9861-26b3fde522ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.408347 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80029da6-415f-4239-9861-26b3fde522ad admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.408627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80029da6-415f-4239-9861-26b3fde522ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.408892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80029da6-415f-4239-9861-26b3fde522ad admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.409524 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80029da6-415f-4239-9861-26b3fde522ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.411177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80029da6-415f-4239-9861-26b3fde522ad admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.421197 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1338/2714] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:04:04.425154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80029da6-415f-4239-9861-26b3fde522ad admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:04.434976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80029da6-415f-4239-9861-26b3fde522ad admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.439752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699768d-cac0-4450-ae1d-df085a77447d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.440335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699768d-cac0-4450-ae1d-df085a77447d admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.440612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699768d-cac0-4450-ae1d-df085a77447d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.440887 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699768d-cac0-4450-ae1d-df085a77447d admin admin] PATH_INFO: `/v3/projects/b95b09a0ea444e25afaec223d35868db/users/4c20b3a4f7c84f06820a923774af1c93/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.458982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1377/2715] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:04:04 2022] POST /identity/v3/projects => generated 359 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:04:04.459172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699768d-cac0-4450-ae1d-df085a77447d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b95b09a0ea444e25afaec223d35868db, user_id=4c20b3a4f7c84f06820a923774af1c93, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.459172 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699768d-cac0-4450-ae1d-df085a77447d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.459951 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cc14f-70a6-4f21-8ccb-11c4823d640a admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:04.520317 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cc14f-70a6-4f21-8ccb-11c4823d640a admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.525660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025ce97-8bf1-45e7-96f8-3cf10d043f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.526262 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025ce97-8bf1-45e7-96f8-3cf10d043f84 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.526568 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025ce97-8bf1-45e7-96f8-3cf10d043f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.526845 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025ce97-8bf1-45e7-96f8-3cf10d043f84 admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.527471 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025ce97-8bf1-45e7-96f8-3cf10d043f84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.528893 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1339/2716] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:04:04 2022] PUT /identity/v3/projects/b95b09a0ea444e25afaec223d35868db/users/4c20b3a4f7c84f06820a923774af1c93/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:04:04.529371 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025ce97-8bf1-45e7-96f8-3cf10d043f84 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.532786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699768d-cac0-4450-ae1d-df085a77447d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:04.596334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699768d-cac0-4450-ae1d-df085a77447d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.600854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e16a9-5199-4395-875a-d0ceff32c3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.601450 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e16a9-5199-4395-875a-d0ceff32c3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.601728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e16a9-5199-4395-875a-d0ceff32c3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.602114 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e16a9-5199-4395-875a-d0ceff32c3e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.602543 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1378/2717] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:04:04 2022] POST /identity/v3/users => generated 280 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:04:04.602783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e16a9-5199-4395-875a-d0ceff32c3e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.604421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e16a9-5199-4395-875a-d0ceff32c3e3 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.606904 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025ce97-8bf1-45e7-96f8-3cf10d043f84 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:04.617376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1340/2718] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:04:04.617576 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025ce97-8bf1-45e7-96f8-3cf10d043f84 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.618788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e16a9-5199-4395-875a-d0ceff32c3e3 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:04.622240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6913a1-c5bf-432a-93ed-b71369917f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.622942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6913a1-c5bf-432a-93ed-b71369917f04 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.623235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6913a1-c5bf-432a-93ed-b71369917f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.623498 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6913a1-c5bf-432a-93ed-b71369917f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.624140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6913a1-c5bf-432a-93ed-b71369917f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.625885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6913a1-c5bf-432a-93ed-b71369917f04 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.628574 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e16a9-5199-4395-875a-d0ceff32c3e3 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.634088 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf78131-39d4-48b5-994f-4157ee8dc884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.634763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf78131-39d4-48b5-994f-4157ee8dc884 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.635046 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf78131-39d4-48b5-994f-4157ee8dc884 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.635348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf78131-39d4-48b5-994f-4157ee8dc884 admin admin] PATH_INFO: `/v3/projects/b95b09a0ea444e25afaec223d35868db/users/4c20b3a4f7c84f06820a923774af1c93/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.637830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1379/2719] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:04:04.637959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf78131-39d4-48b5-994f-4157ee8dc884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b95b09a0ea444e25afaec223d35868db, user_id=4c20b3a4f7c84f06820a923774af1c93, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.640025 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf78131-39d4-48b5-994f-4157ee8dc884 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.644420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6913a1-c5bf-432a-93ed-b71369917f04 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:04.651765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1341/2720] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:04:04 2022] PUT /identity/v3/projects/b95b09a0ea444e25afaec223d35868db/users/4c20b3a4f7c84f06820a923774af1c93/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:04.655220 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6913a1-c5bf-432a-93ed-b71369917f04 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.661691 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387193e2-4d97-40f3-b9cc-4c31f49c7f7a None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.661984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387193e2-4d97-40f3-b9cc-4c31f49c7f7a None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.662259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387193e2-4d97-40f3-b9cc-4c31f49c7f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.683370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:04.691686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-387193e2-4d97-40f3-b9cc-4c31f49c7f7a None None] MFA Rules not processed for user `4c20b3a4f7c84f06820a923774af1c93`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:04.692905 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-387193e2-4d97-40f3-b9cc-4c31f49c7f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:04.707351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:04.716261 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8d314-80e0-423e-ad4e-92a96a1a79f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.716868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8d314-80e0-423e-ad4e-92a96a1a79f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.717140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8d314-80e0-423e-ad4e-92a96a1a79f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.717404 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8d314-80e0-423e-ad4e-92a96a1a79f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.718110 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8d314-80e0-423e-ad4e-92a96a1a79f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.719695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8d314-80e0-423e-ad4e-92a96a1a79f5 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.730075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1380/2721] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:04:04.734072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8d314-80e0-423e-ad4e-92a96a1a79f5 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:04.743598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8d314-80e0-423e-ad4e-92a96a1a79f5 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.748144 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9067f86-6565-4825-a2f6-b1c1ed497418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.749383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9067f86-6565-4825-a2f6-b1c1ed497418 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.749383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9067f86-6565-4825-a2f6-b1c1ed497418 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.749383 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9067f86-6565-4825-a2f6-b1c1ed497418 admin admin] PATH_INFO: `/v3/projects/3c9582a480334342861b6698b5828c3f/users/9abf81e5e037446aa8da0bda5194dc45/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.761417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1342/2722] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:04:04.764008 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9067f86-6565-4825-a2f6-b1c1ed497418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9582a480334342861b6698b5828c3f, user_id=9abf81e5e037446aa8da0bda5194dc45, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.766242 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9067f86-6565-4825-a2f6-b1c1ed497418 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.773302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387193e2-4d97-40f3-b9cc-4c31f49c7f7a None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:04.775325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-387193e2-4d97-40f3-b9cc-4c31f49c7f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:04.830502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387193e2-4d97-40f3-b9cc-4c31f49c7f7a None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:04.834395 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957b48e-a9c0-4d25-8085-fe736a000605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:04.835019 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957b48e-a9c0-4d25-8085-fe736a000605 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.835294 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957b48e-a9c0-4d25-8085-fe736a000605 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.835563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957b48e-a9c0-4d25-8085-fe736a000605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.837360 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2957b48e-a9c0-4d25-8085-fe736a000605 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:04.863374 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1381/2723] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:04:04 2022] PUT /identity/v3/projects/3c9582a480334342861b6698b5828c3f/users/9abf81e5e037446aa8da0bda5194dc45/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:04:04.899141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957b48e-a9c0-4d25-8085-fe736a000605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:04.900658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957b48e-a9c0-4d25-8085-fe736a000605 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:04.911228 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1343/2724] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:04:04.930791 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87305f-0d8d-4249-a0d7-560c93666c7f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:04.931279 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87305f-0d8d-4249-a0d7-560c93666c7f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:04.931666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87305f-0d8d-4249-a0d7-560c93666c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:04.960240 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-aa87305f-0d8d-4249-a0d7-560c93666c7f None None] MFA Rules not processed for user `9abf81e5e037446aa8da0bda5194dc45`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:04.961581 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-aa87305f-0d8d-4249-a0d7-560c93666c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:05.025862 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1382/2725] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:05.060725 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957b48e-a9c0-4d25-8085-fe736a000605 service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:05.062133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2957b48e-a9c0-4d25-8085-fe736a000605 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:05.073814 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957b48e-a9c0-4d25-8085-fe736a000605 service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:05.119282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:05.127298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa36e1-2e94-48c0-b0a3-bde0793ed127 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:05.139836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa36e1-2e94-48c0-b0a3-bde0793ed127 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:05.139836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa36e1-2e94-48c0-b0a3-bde0793ed127 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:05.139836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa36e1-2e94-48c0-b0a3-bde0793ed127 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:05.141272 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa36e1-2e94-48c0-b0a3-bde0793ed127 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:05.142852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa36e1-2e94-48c0-b0a3-bde0793ed127 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:05.153913 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1344/2726] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:04:05.169876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87305f-0d8d-4249-a0d7-560c93666c7f None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:05.232720 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87305f-0d8d-4249-a0d7-560c93666c7f None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:05.237217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48459487-b951-4c22-900c-606498f71940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:05.237833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48459487-b951-4c22-900c-606498f71940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:05.238108 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48459487-b951-4c22-900c-606498f71940 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:05.238377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48459487-b951-4c22-900c-606498f71940 admin admin] PATH_INFO: `/v3/users/9abf81e5e037446aa8da0bda5194dc45` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:05.239553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48459487-b951-4c22-900c-606498f71940 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9abf81e5e037446aa8da0bda5194dc45)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:05.241413 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48459487-b951-4c22-900c-606498f71940 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:05.458206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-48459487-b951-4c22-900c-606498f71940 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9abf81e5e037446aa8da0bda5194dc45'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:05.527698 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-48459487-b951-4c22-900c-606498f71940 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9abf81e5e037446aa8da0bda5194dc45'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:05.528485 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-48459487-b951-4c22-900c-606498f71940 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9abf81e5e037446aa8da0bda5194dc45'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:05.529068 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-48459487-b951-4c22-900c-606498f71940 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9abf81e5e037446aa8da0bda5194dc45'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:05.531795 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:05.543425 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-48459487-b951-4c22-900c-606498f71940 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9abf81e5e037446aa8da0bda5194dc45'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:05.556373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1383/2727] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:04:05 2022] DELETE /identity/v3/users/9abf81e5e037446aa8da0bda5194dc45 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:05.560934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f433d28-af50-48ee-bf81-57ca3154f139 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:05.561387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f433d28-af50-48ee-bf81-57ca3154f139 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:05.561492 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f433d28-af50-48ee-bf81-57ca3154f139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:05.581892 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-0f433d28-af50-48ee-bf81-57ca3154f139 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:05.583475 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0f433d28-af50-48ee-bf81-57ca3154f139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:05.643648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1345/2728] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:05.655648 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48459487-b951-4c22-900c-606498f71940 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:05.657598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-48459487-b951-4c22-900c-606498f71940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:05.730234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48459487-b951-4c22-900c-606498f71940 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:05.735003 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72f59b-1a98-4aa2-b01a-642674b65c19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:05.735604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72f59b-1a98-4aa2-b01a-642674b65c19 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:05.735879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72f59b-1a98-4aa2-b01a-642674b65c19 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:05.736145 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72f59b-1a98-4aa2-b01a-642674b65c19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:05.745964 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72f59b-1a98-4aa2-b01a-642674b65c19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:05.749509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72f59b-1a98-4aa2-b01a-642674b65c19 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:05.756998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1384/2729] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:04:06.132140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f433d28-af50-48ee-bf81-57ca3154f139 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:06.134313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0f433d28-af50-48ee-bf81-57ca3154f139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:06.146836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f433d28-af50-48ee-bf81-57ca3154f139 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:06.152386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ee012-cc08-4ed4-8ce3-d1658696e64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:06.153091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ee012-cc08-4ed4-8ce3-d1658696e64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:06.153460 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ee012-cc08-4ed4-8ce3-d1658696e64e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:06.153634 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ee012-cc08-4ed4-8ce3-d1658696e64e admin admin] PATH_INFO: `/v3/projects/3c9582a480334342861b6698b5828c3f` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:06.154980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ee012-cc08-4ed4-8ce3-d1658696e64e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c9582a480334342861b6698b5828c3f)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:06.157592 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ee012-cc08-4ed4-8ce3-d1658696e64e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:06.265536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a00ee012-cc08-4ed4-8ce3-d1658696e64e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9582a480334342861b6698b5828c3f', 'request_id': 'req-a00ee012-cc08-4ed4-8ce3-d1658696e64e'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:06.271529 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a00ee012-cc08-4ed4-8ce3-d1658696e64e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9582a480334342861b6698b5828c3f', 'request_id': 'req-a00ee012-cc08-4ed4-8ce3-d1658696e64e'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:06.333694 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a00ee012-cc08-4ed4-8ce3-d1658696e64e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9582a480334342861b6698b5828c3f', 'request_id': 'req-a00ee012-cc08-4ed4-8ce3-d1658696e64e'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:06.345060 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a00ee012-cc08-4ed4-8ce3-d1658696e64e admin admin] The token cache is being invalidate because project 3c9582a480334342861b6698b5828c3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:04:06.345346 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a00ee012-cc08-4ed4-8ce3-d1658696e64e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:06.346024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a00ee012-cc08-4ed4-8ce3-d1658696e64e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:06.347524 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1346/2730] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:04:06 2022] DELETE /identity/v3/projects/3c9582a480334342861b6698b5828c3f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:06.352943 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e1dad-950c-4dd4-97b5-769e8a9e961a None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:06.353428 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e1dad-950c-4dd4-97b5-769e8a9e961a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:06.353915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e1dad-950c-4dd4-97b5-769e8a9e961a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:06.374500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-bc1e1dad-950c-4dd4-97b5-769e8a9e961a None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:06.375759 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-bc1e1dad-950c-4dd4-97b5-769e8a9e961a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:06.435355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1385/2731] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:06.444838 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7396ff-e93c-4421-8ae0-99da28f82235 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:06.445294 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7396ff-e93c-4421-8ae0-99da28f82235 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:06.446384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7396ff-e93c-4421-8ae0-99da28f82235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:06.469681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-6e7396ff-e93c-4421-8ae0-99da28f82235 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:06.470937 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6e7396ff-e93c-4421-8ae0-99da28f82235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:06.478962 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1347/2732] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:06.483222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e1dad-950c-4dd4-97b5-769e8a9e961a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:06.493268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e1dad-950c-4dd4-97b5-769e8a9e961a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:06.497770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932d6a0-de19-4846-ad3a-6a3fcd11ac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:06.498350 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932d6a0-de19-4846-ad3a-6a3fcd11ac4f admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:06.498655 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932d6a0-de19-4846-ad3a-6a3fcd11ac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:06.498923 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932d6a0-de19-4846-ad3a-6a3fcd11ac4f admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:06.499555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932d6a0-de19-4846-ad3a-6a3fcd11ac4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:04:06.499854 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932d6a0-de19-4846-ad3a-6a3fcd11ac4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:06.501397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932d6a0-de19-4846-ad3a-6a3fcd11ac4f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:06.513114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1386/2733] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:04:06.517722 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7396ff-e93c-4421-8ae0-99da28f82235 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:06.527012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7396ff-e93c-4421-8ae0-99da28f82235 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:06.531553 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9785a-3498-45f6-9cf8-ba7ff3466425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:06.532139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9785a-3498-45f6-9cf8-ba7ff3466425 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:06.532418 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9785a-3498-45f6-9cf8-ba7ff3466425 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:06.532688 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9785a-3498-45f6-9cf8-ba7ff3466425 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:06.533315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9785a-3498-45f6-9cf8-ba7ff3466425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:06.535233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9785a-3498-45f6-9cf8-ba7ff3466425 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:06.644334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1348/2734] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:04:06 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:04:06.649875 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932d6a0-de19-4846-ad3a-6a3fcd11ac4f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:06.660522 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932d6a0-de19-4846-ad3a-6a3fcd11ac4f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:06.719001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf91dde-6f79-4017-895a-0ea8e661ff7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:06.719606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf91dde-6f79-4017-895a-0ea8e661ff7b admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:06.719885 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf91dde-6f79-4017-895a-0ea8e661ff7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:06.720155 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf91dde-6f79-4017-895a-0ea8e661ff7b admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:06.720788 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf91dde-6f79-4017-895a-0ea8e661ff7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:06.722705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf91dde-6f79-4017-895a-0ea8e661ff7b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:06.733048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:06.820176 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1387/2735] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:04:06 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:04:06.825016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9785a-3498-45f6-9cf8-ba7ff3466425 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:06.837156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9785a-3498-45f6-9cf8-ba7ff3466425 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:06.842666 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61608f-acd5-49cb-92c8-d25c0c66747a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:06.842666 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61608f-acd5-49cb-92c8-d25c0c66747a admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:06.842894 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61608f-acd5-49cb-92c8-d25c0c66747a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:06.842984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61608f-acd5-49cb-92c8-d25c0c66747a admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:06.843575 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61608f-acd5-49cb-92c8-d25c0c66747a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:06.845141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61608f-acd5-49cb-92c8-d25c0c66747a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:06.856407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1349/2736] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:04:06.860306 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf91dde-6f79-4017-895a-0ea8e661ff7b admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:06.860834 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61608f-acd5-49cb-92c8-d25c0c66747a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:06.862865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-8e61608f-acd5-49cb-92c8-d25c0c66747a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:06.873275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf91dde-6f79-4017-895a-0ea8e661ff7b admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:06.877899 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27d203-6e80-44b4-a03d-89a1e09b485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:06.878415 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27d203-6e80-44b4-a03d-89a1e09b485c admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:06.878743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27d203-6e80-44b4-a03d-89a1e09b485c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:06.879012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27d203-6e80-44b4-a03d-89a1e09b485c admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:06.879749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f27d203-6e80-44b4-a03d-89a1e09b485c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:06.881343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f27d203-6e80-44b4-a03d-89a1e09b485c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:06.892536 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1388/2737] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:04:06.896945 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27d203-6e80-44b4-a03d-89a1e09b485c admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:06.904242 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:06.906712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27d203-6e80-44b4-a03d-89a1e09b485c admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:06.911597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ceb2c9-5e0b-4f5b-a5fd-d32bb04ba490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:06.912222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ceb2c9-5e0b-4f5b-a5fd-d32bb04ba490 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:06.912498 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ceb2c9-5e0b-4f5b-a5fd-d32bb04ba490 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:06.912764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ceb2c9-5e0b-4f5b-a5fd-d32bb04ba490 admin admin] PATH_INFO: `/v3/projects/6e6cb41b3ee34e6a8418bc61f712e12b/users/8e00b9fe61f3436eac7460ab871e586c/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:06.922770 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61608f-acd5-49cb-92c8-d25c0c66747a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:06.927070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc44187-ec22-433b-9b72-afbd676e0a9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:06.927744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc44187-ec22-433b-9b72-afbd676e0a9f service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:06.928053 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ceb2c9-5e0b-4f5b-a5fd-d32bb04ba490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e6cb41b3ee34e6a8418bc61f712e12b, user_id=8e00b9fe61f3436eac7460ab871e586c, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:06.928170 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc44187-ec22-433b-9b72-afbd676e0a9f service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:06.928707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc44187-ec22-433b-9b72-afbd676e0a9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:06.931256 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ceb2c9-5e0b-4f5b-a5fd-d32bb04ba490 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:06.931715 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7fc44187-ec22-433b-9b72-afbd676e0a9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:06.990369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc44187-ec22-433b-9b72-afbd676e0a9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:06.991845 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc44187-ec22-433b-9b72-afbd676e0a9f service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:06.998175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1389/2738] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:04:06 2022] PUT /identity/v3/projects/6e6cb41b3ee34e6a8418bc61f712e12b/users/8e00b9fe61f3436eac7460ab871e586c/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:04:07.001822 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1350/2739] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:04:07.008426 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465c137-840f-4513-b98f-78093a49a129 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:07.008707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465c137-840f-4513-b98f-78093a49a129 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:07.008979 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465c137-840f-4513-b98f-78093a49a129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:07.039298 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-2465c137-840f-4513-b98f-78093a49a129 None None] MFA Rules not processed for user `8e00b9fe61f3436eac7460ab871e586c`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:07.040953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2465c137-840f-4513-b98f-78093a49a129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:07.113061 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1390/2740] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:07.124339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc44187-ec22-433b-9b72-afbd676e0a9f service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:07.125733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7fc44187-ec22-433b-9b72-afbd676e0a9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:07.136971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc44187-ec22-433b-9b72-afbd676e0a9f service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:07.191641 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3355b-a1d0-4f1a-9c15-a39180106d02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:07.192279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3355b-a1d0-4f1a-9c15-a39180106d02 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:07.192564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3355b-a1d0-4f1a-9c15-a39180106d02 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:07.192877 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3355b-a1d0-4f1a-9c15-a39180106d02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:07.201726 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3355b-a1d0-4f1a-9c15-a39180106d02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:07.203183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3355b-a1d0-4f1a-9c15-a39180106d02 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:07.214018 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1351/2741] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:04:08.436147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465c137-840f-4513-b98f-78093a49a129 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:08.445775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465c137-840f-4513-b98f-78093a49a129 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:08.449338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df526-4408-47d3-b247-f7388b3ab81d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:08.449931 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df526-4408-47d3-b247-f7388b3ab81d service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:08.450212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df526-4408-47d3-b247-f7388b3ab81d service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:08.450506 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df526-4408-47d3-b247-f7388b3ab81d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:08.452212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7b5df526-4408-47d3-b247-f7388b3ab81d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:08.513730 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5df526-4408-47d3-b247-f7388b3ab81d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:08.515183 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5df526-4408-47d3-b247-f7388b3ab81d service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:08.525140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1391/2742] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:04:09.007816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709167b8-1766-4797-b60a-8fad340cdd0f None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:09.008899 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709167b8-1766-4797-b60a-8fad340cdd0f None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:09.008899 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709167b8-1766-4797-b60a-8fad340cdd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:09.028068 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-709167b8-1766-4797-b60a-8fad340cdd0f None None] MFA Rules not processed for user `8e00b9fe61f3436eac7460ab871e586c`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:09.029260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-709167b8-1766-4797-b60a-8fad340cdd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:09.036072 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1352/2743] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:13.069402 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df526-4408-47d3-b247-f7388b3ab81d service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:13.071238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7b5df526-4408-47d3-b247-f7388b3ab81d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:13.130133 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:13.149930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df526-4408-47d3-b247-f7388b3ab81d service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:13.154461 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:13.155122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:13.155421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:13.155678 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] PATH_INFO: `/v3/users/cd7fdf9c3d0c444e987e2a0873b6602b` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:13.156813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd7fdf9c3d0c444e987e2a0873b6602b)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:13.158751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:13.319820 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd7fdf9c3d0c444e987e2a0873b6602b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:13.402867 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd7fdf9c3d0c444e987e2a0873b6602b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:13.406822 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd7fdf9c3d0c444e987e2a0873b6602b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:13.406822 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd7fdf9c3d0c444e987e2a0873b6602b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:13.423037 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd7fdf9c3d0c444e987e2a0873b6602b'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:13.438362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1392/2744] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:04:13 2022] DELETE /identity/v3/users/cd7fdf9c3d0c444e987e2a0873b6602b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:13.442909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699c4e2-b056-40ac-90a8-6329e320b51c None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:13.443437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699c4e2-b056-40ac-90a8-6329e320b51c None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:13.443437 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699c4e2-b056-40ac-90a8-6329e320b51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:13.462658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-4699c4e2-b056-40ac-90a8-6329e320b51c None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:13.463902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-4699c4e2-b056-40ac-90a8-6329e320b51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:13.522763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1353/2745] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:04:13.532840 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:13.534246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:13.596113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023238b1-5b47-4903-b43e-c3d429a36f5b admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:13.600024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52281a73-74a6-4d45-b22b-5e25ffe5eff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:13.600745 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52281a73-74a6-4d45-b22b-5e25ffe5eff7 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:13.601025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52281a73-74a6-4d45-b22b-5e25ffe5eff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:13.601295 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52281a73-74a6-4d45-b22b-5e25ffe5eff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:13.610658 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52281a73-74a6-4d45-b22b-5e25ffe5eff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:13.612202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52281a73-74a6-4d45-b22b-5e25ffe5eff7 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:13.622287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1393/2746] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:04:14.101540 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699c4e2-b056-40ac-90a8-6329e320b51c None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:14.104125 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-4699c4e2-b056-40ac-90a8-6329e320b51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:14.117276 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699c4e2-b056-40ac-90a8-6329e320b51c None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:14.128075 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:14.128254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:14.128538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:14.128813 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] PATH_INFO: `/v3/projects/c5d9325889ba4e55904ddea35f68dd33` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:14.129952 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5d9325889ba4e55904ddea35f68dd33)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:14.131956 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:14.172111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=100 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:14.257819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5d9325889ba4e55904ddea35f68dd33', 'request_id': 'req-f6aae978-1985-4dd8-8b32-df7c5d898e97'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:14.265511 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5d9325889ba4e55904ddea35f68dd33', 'request_id': 'req-f6aae978-1985-4dd8-8b32-df7c5d898e97'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:14.373934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5d9325889ba4e55904ddea35f68dd33', 'request_id': 'req-f6aae978-1985-4dd8-8b32-df7c5d898e97'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:14.385674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] The token cache is being invalidate because project c5d9325889ba4e55904ddea35f68dd33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:04:14.385955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:14.386626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:14.388229 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1354/2747] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:04:14 2022] DELETE /identity/v3/projects/c5d9325889ba4e55904ddea35f68dd33 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:14.392271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52281a73-74a6-4d45-b22b-5e25ffe5eff7 service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:14.403192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52281a73-74a6-4d45-b22b-5e25ffe5eff7 service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:14.459484 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:14.460116 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:14.460392 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:14.460663 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] PATH_INFO: `/v3/users/30cd98d792b24217b1efa3934d1fd081` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:14.461895 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=30cd98d792b24217b1efa3934d1fd081)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:14.465466 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:14.470238 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:14.638831 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30cd98d792b24217b1efa3934d1fd081'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:14.704688 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30cd98d792b24217b1efa3934d1fd081'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:14.705300 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30cd98d792b24217b1efa3934d1fd081'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:14.705932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30cd98d792b24217b1efa3934d1fd081'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:14.719007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30cd98d792b24217b1efa3934d1fd081'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:14.731087 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1394/2748] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:04:14 2022] DELETE /identity/v3/users/30cd98d792b24217b1efa3934d1fd081 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:15.153579 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:15.155122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:15.169884 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aae978-1985-4dd8-8b32-df7c5d898e97 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:15.228138 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:15.228753 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:15.229029 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:15.229299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] PATH_INFO: `/v3/projects/766860b5768240e992174761f07b4142` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:15.230400 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=766860b5768240e992174761f07b4142)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:15.232366 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:15.329578 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '766860b5768240e992174761f07b4142', 'request_id': 'req-fdc89196-2e76-4832-956c-fe5476a309ef'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:15.336042 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '766860b5768240e992174761f07b4142', 'request_id': 'req-fdc89196-2e76-4832-956c-fe5476a309ef'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:15.402449 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '766860b5768240e992174761f07b4142', 'request_id': 'req-fdc89196-2e76-4832-956c-fe5476a309ef'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:15.414047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] The token cache is being invalidate because project 766860b5768240e992174761f07b4142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:04:15.414390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:15.415128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:15.416677 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1355/2749] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:04:15 2022] DELETE /identity/v3/projects/766860b5768240e992174761f07b4142 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:15.421513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:15.422802 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:15.437381 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc563aee-b491-414f-bbc3-71f1347684fc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:15.497129 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:15.497755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:15.498042 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:15.498318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] PATH_INFO: `/v3/users/4c20b3a4f7c84f06820a923774af1c93` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:15.499478 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c20b3a4f7c84f06820a923774af1c93)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:15.501365 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:15.643513 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c20b3a4f7c84f06820a923774af1c93'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:15.710896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c20b3a4f7c84f06820a923774af1c93'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:15.711451 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c20b3a4f7c84f06820a923774af1c93'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:15.712055 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c20b3a4f7c84f06820a923774af1c93'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:15.724556 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c20b3a4f7c84f06820a923774af1c93'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:15.737077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1395/2750] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:04:15 2022] DELETE /identity/v3/users/4c20b3a4f7c84f06820a923774af1c93 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:16.095666 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:16.100961 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:16.113260 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc89196-2e76-4832-956c-fe5476a309ef admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:16.153271 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:16.171477 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9203b-94c5-4368-9516-bb4c0507890e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:16.172140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9203b-94c5-4368-9516-bb4c0507890e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:16.172417 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9203b-94c5-4368-9516-bb4c0507890e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:16.172684 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9203b-94c5-4368-9516-bb4c0507890e admin admin] PATH_INFO: `/v3/projects/b95b09a0ea444e25afaec223d35868db` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:16.173815 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9203b-94c5-4368-9516-bb4c0507890e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b95b09a0ea444e25afaec223d35868db)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:16.175735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9203b-94c5-4368-9516-bb4c0507890e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:16.275972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b5b9203b-94c5-4368-9516-bb4c0507890e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b95b09a0ea444e25afaec223d35868db', 'request_id': 'req-b5b9203b-94c5-4368-9516-bb4c0507890e'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:16.281940 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b5b9203b-94c5-4368-9516-bb4c0507890e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b95b09a0ea444e25afaec223d35868db', 'request_id': 'req-b5b9203b-94c5-4368-9516-bb4c0507890e'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:16.356380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b5b9203b-94c5-4368-9516-bb4c0507890e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b95b09a0ea444e25afaec223d35868db', 'request_id': 'req-b5b9203b-94c5-4368-9516-bb4c0507890e'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:16.367253 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b5b9203b-94c5-4368-9516-bb4c0507890e admin admin] The token cache is being invalidate because project b95b09a0ea444e25afaec223d35868db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:04:16.367604 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b5b9203b-94c5-4368-9516-bb4c0507890e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:16.368175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-b5b9203b-94c5-4368-9516-bb4c0507890e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:16.370299 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1356/2751] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:04:16 2022] DELETE /identity/v3/projects/b95b09a0ea444e25afaec223d35868db => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:20.437766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:20.439180 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:20.444450 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:20.497716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f052d33-fbb6-4975-9c87-5d2367b748e1 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:20.544500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf62181-4216-4eee-907f-71b7bc0d19a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:20.545362 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf62181-4216-4eee-907f-71b7bc0d19a0 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:20.545882 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf62181-4216-4eee-907f-71b7bc0d19a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:20.546775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf62181-4216-4eee-907f-71b7bc0d19a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:20.548496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-cbf62181-4216-4eee-907f-71b7bc0d19a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:20.606828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf62181-4216-4eee-907f-71b7bc0d19a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:20.608248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf62181-4216-4eee-907f-71b7bc0d19a0 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:20.617001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1396/2752] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:04:26.364008 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=33,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:27.819064 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a8c0f-d2e2-4028-b5f2-7ec5db59622c None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:27.819640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a8c0f-d2e2-4028-b5f2-7ec5db59622c None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:27.819640 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a8c0f-d2e2-4028-b5f2-7ec5db59622c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:27.848289 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-5a4a8c0f-d2e2-4028-b5f2-7ec5db59622c None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:27.849471 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5a4a8c0f-d2e2-4028-b5f2-7ec5db59622c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:27.908839 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1357/2753] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 13:04:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:28.036522 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf62181-4216-4eee-907f-71b7bc0d19a0 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:28.037917 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-cbf62181-4216-4eee-907f-71b7bc0d19a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:28.108180 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf62181-4216-4eee-907f-71b7bc0d19a0 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:28.115308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a097f-7a54-4d5d-9e2a-3e22f5ae0c7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:28.124247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a097f-7a54-4d5d-9e2a-3e22f5ae0c7c service swift] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:28.124247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a097f-7a54-4d5d-9e2a-3e22f5ae0c7c service swift] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:28.124247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a097f-7a54-4d5d-9e2a-3e22f5ae0c7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:28.130553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a097f-7a54-4d5d-9e2a-3e22f5ae0c7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:28.133215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a097f-7a54-4d5d-9e2a-3e22f5ae0c7c service swift] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:28.146166 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1397/2754] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 13:04:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:04:28.953040 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb7d40-e6a1-452d-a151-846d0050b7f4 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:28.953514 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb7d40-e6a1-452d-a151-846d0050b7f4 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:28.953610 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb7d40-e6a1-452d-a151-846d0050b7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:28.972513 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-52cb7d40-e6a1-452d-a151-846d0050b7f4 None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:28.973698 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-52cb7d40-e6a1-452d-a151-846d0050b7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:28.980849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1358/2755] 10.222.0.46 () {58 vars in 967 bytes} [Fri Sep 9 13:04:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:28.992751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a097f-7a54-4d5d-9e2a-3e22f5ae0c7c service swift] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:28.998695 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:29.004367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a097f-7a54-4d5d-9e2a-3e22f5ae0c7c service swift] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:29.009404 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9fadd-0988-47e4-b596-85a38d4b9594 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:29.009984 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9fadd-0988-47e4-b596-85a38d4b9594 service swift] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:29.010259 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9fadd-0988-47e4-b596-85a38d4b9594 service swift] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:29.010869 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9fadd-0988-47e4-b596-85a38d4b9594 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:29.018716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9fadd-0988-47e4-b596-85a38d4b9594 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:29.020324 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9fadd-0988-47e4-b596-85a38d4b9594 service swift] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:29.029083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1398/2756] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 13:04:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:04:29.176033 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94da905-99fb-4702-bb75-da9d69cdebb0 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:29.176237 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94da905-99fb-4702-bb75-da9d69cdebb0 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:29.176513 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94da905-99fb-4702-bb75-da9d69cdebb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:29.196780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-b94da905-99fb-4702-bb75-da9d69cdebb0 None None] MFA Rules not processed for user `4c96322c48c146c7b8bcb9959f533820`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:29.198650 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b94da905-99fb-4702-bb75-da9d69cdebb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:29.205881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1359/2757] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:29.718153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861f8a-8f69-4687-bc27-25dfb9c196ab None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:29.721388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861f8a-8f69-4687-bc27-25dfb9c196ab None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:29.721672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861f8a-8f69-4687-bc27-25dfb9c196ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:29.742780 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-5f861f8a-8f69-4687-bc27-25dfb9c196ab None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:29.745232 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5f861f8a-8f69-4687-bc27-25dfb9c196ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:29.751631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1399/2758] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 13:04:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:29.765930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94da905-99fb-4702-bb75-da9d69cdebb0 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:29.780051 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94da905-99fb-4702-bb75-da9d69cdebb0 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:29.788283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39940abb-73f3-4562-8e88-deee345f7a93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:29.789173 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39940abb-73f3-4562-8e88-deee345f7a93 service swift] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:29.789599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39940abb-73f3-4562-8e88-deee345f7a93 service swift] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:29.790105 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39940abb-73f3-4562-8e88-deee345f7a93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:29.800364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39940abb-73f3-4562-8e88-deee345f7a93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:29.802752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39940abb-73f3-4562-8e88-deee345f7a93 service swift] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:29.812406 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1360/2759] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 13:04:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:04:30.511274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43563c8f-1e1c-4378-9f6e-68162581a1da None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:30.511274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43563c8f-1e1c-4378-9f6e-68162581a1da None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:30.511274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43563c8f-1e1c-4378-9f6e-68162581a1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:30.536080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-43563c8f-1e1c-4378-9f6e-68162581a1da None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:30.537889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-43563c8f-1e1c-4378-9f6e-68162581a1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:30.555304 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1400/2760] 10.222.0.46 () {58 vars in 967 bytes} [Fri Sep 9 13:04:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:30.571984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39940abb-73f3-4562-8e88-deee345f7a93 service swift] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:30.585828 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39940abb-73f3-4562-8e88-deee345f7a93 service swift] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:30.592856 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e522b3-a0f7-450f-93b9-3f25a6b00b14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:30.593584 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e522b3-a0f7-450f-93b9-3f25a6b00b14 service swift] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:30.593964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e522b3-a0f7-450f-93b9-3f25a6b00b14 service swift] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:30.594347 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e522b3-a0f7-450f-93b9-3f25a6b00b14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:30.605789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e522b3-a0f7-450f-93b9-3f25a6b00b14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:30.608392 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e522b3-a0f7-450f-93b9-3f25a6b00b14 service swift] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:30.618866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1361/2761] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 13:04:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:04:33.432881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43563c8f-1e1c-4378-9f6e-68162581a1da None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:33.434315 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-43563c8f-1e1c-4378-9f6e-68162581a1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:33.505075 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43563c8f-1e1c-4378-9f6e-68162581a1da None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:33.509766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:33.510435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:33.510808 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:33.511146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] PATH_INFO: `/v3/users/8e00b9fe61f3436eac7460ab871e586c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:33.512342 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e00b9fe61f3436eac7460ab871e586c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:33.514289 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:33.725683 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e00b9fe61f3436eac7460ab871e586c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:33.796902 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e00b9fe61f3436eac7460ab871e586c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:33.797601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e00b9fe61f3436eac7460ab871e586c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:33.798275 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e00b9fe61f3436eac7460ab871e586c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:33.816398 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e00b9fe61f3436eac7460ab871e586c'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:33.838127 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1401/2762] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:04:33 2022] DELETE /identity/v3/users/8e00b9fe61f3436eac7460ab871e586c => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:33.843043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf42d57-9a26-4e87-bb63-0694dd2d1789 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:33.845985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf42d57-9a26-4e87-bb63-0694dd2d1789 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:33.845985 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf42d57-9a26-4e87-bb63-0694dd2d1789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:33.863270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-baf42d57-9a26-4e87-bb63-0694dd2d1789 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:33.864715 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-baf42d57-9a26-4e87-bb63-0694dd2d1789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:33.926633 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1362/2763] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:33.936809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:33.938323 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:34.000058 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0cd19-0f57-4b52-acfc-ab783bdb6308 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:34.004022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59a1fa-4974-427a-8233-af8c4764fad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:34.004657 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59a1fa-4974-427a-8233-af8c4764fad0 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:34.005334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59a1fa-4974-427a-8233-af8c4764fad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:34.005334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59a1fa-4974-427a-8233-af8c4764fad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:34.015351 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59a1fa-4974-427a-8233-af8c4764fad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:34.017900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59a1fa-4974-427a-8233-af8c4764fad0 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:34.020871 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:34.028107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1402/2764] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:04:34.263336 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf42d57-9a26-4e87-bb63-0694dd2d1789 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:34.264761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-baf42d57-9a26-4e87-bb63-0694dd2d1789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:34.277367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf42d57-9a26-4e87-bb63-0694dd2d1789 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:34.282074 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76975de7-26c8-482e-bb94-6dc4a9bef53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:34.282949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76975de7-26c8-482e-bb94-6dc4a9bef53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:34.283691 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76975de7-26c8-482e-bb94-6dc4a9bef53a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:34.283691 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76975de7-26c8-482e-bb94-6dc4a9bef53a admin admin] PATH_INFO: `/v3/projects/6e6cb41b3ee34e6a8418bc61f712e12b` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:34.285231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76975de7-26c8-482e-bb94-6dc4a9bef53a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e6cb41b3ee34e6a8418bc61f712e12b)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:34.287294 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76975de7-26c8-482e-bb94-6dc4a9bef53a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:34.406700 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:34.409633 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-76975de7-26c8-482e-bb94-6dc4a9bef53a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e6cb41b3ee34e6a8418bc61f712e12b', 'request_id': 'req-76975de7-26c8-482e-bb94-6dc4a9bef53a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:34.416122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-76975de7-26c8-482e-bb94-6dc4a9bef53a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e6cb41b3ee34e6a8418bc61f712e12b', 'request_id': 'req-76975de7-26c8-482e-bb94-6dc4a9bef53a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:34.521808 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-76975de7-26c8-482e-bb94-6dc4a9bef53a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e6cb41b3ee34e6a8418bc61f712e12b', 'request_id': 'req-76975de7-26c8-482e-bb94-6dc4a9bef53a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:34.533532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-76975de7-26c8-482e-bb94-6dc4a9bef53a admin admin] The token cache is being invalidate because project 6e6cb41b3ee34e6a8418bc61f712e12b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:04:34.533819 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-76975de7-26c8-482e-bb94-6dc4a9bef53a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:34.534469 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-76975de7-26c8-482e-bb94-6dc4a9bef53a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:04:34.536003 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1363/2765] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:04:34 2022] DELETE /identity/v3/projects/6e6cb41b3ee34e6a8418bc61f712e12b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:34.540771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe973482-b298-4c18-bf05-a2d3466c2b20 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:34.541257 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe973482-b298-4c18-bf05-a2d3466c2b20 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:34.541372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe973482-b298-4c18-bf05-a2d3466c2b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:34.560540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-fe973482-b298-4c18-bf05-a2d3466c2b20 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:34.561726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-fe973482-b298-4c18-bf05-a2d3466c2b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:34.622039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1403/2766] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:34.629871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d02d2a-6c7d-401c-a4bf-c2dc141062c4 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:34.630557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d02d2a-6c7d-401c-a4bf-c2dc141062c4 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:34.630999 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d02d2a-6c7d-401c-a4bf-c2dc141062c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:34.662133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-36d02d2a-6c7d-401c-a4bf-c2dc141062c4 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:34.663747 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-36d02d2a-6c7d-401c-a4bf-c2dc141062c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:34.674291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1364/2767] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:34.678967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe973482-b298-4c18-bf05-a2d3466c2b20 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:34.689454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe973482-b298-4c18-bf05-a2d3466c2b20 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:34.694093 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746474ca-b134-41ce-99ee-ddf06a1521f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:34.694738 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746474ca-b134-41ce-99ee-ddf06a1521f0 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:34.695021 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746474ca-b134-41ce-99ee-ddf06a1521f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:34.695285 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746474ca-b134-41ce-99ee-ddf06a1521f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:34.695932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746474ca-b134-41ce-99ee-ddf06a1521f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:04:34.696226 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746474ca-b134-41ce-99ee-ddf06a1521f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:34.697787 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746474ca-b134-41ce-99ee-ddf06a1521f0 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:34.710867 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1404/2768] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:04:34.715855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d02d2a-6c7d-401c-a4bf-c2dc141062c4 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:34.725327 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d02d2a-6c7d-401c-a4bf-c2dc141062c4 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:34.730206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165548d4-4db6-40b3-8cca-10ba9f42de1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:34.730954 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165548d4-4db6-40b3-8cca-10ba9f42de1b admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:34.731235 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165548d4-4db6-40b3-8cca-10ba9f42de1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:34.733999 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165548d4-4db6-40b3-8cca-10ba9f42de1b admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:34.733999 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165548d4-4db6-40b3-8cca-10ba9f42de1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:34.734900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165548d4-4db6-40b3-8cca-10ba9f42de1b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:34.815489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1365/2769] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:04:34 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:04:34.820050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746474ca-b134-41ce-99ee-ddf06a1521f0 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:34.830826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746474ca-b134-41ce-99ee-ddf06a1521f0 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:34.891277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c88c53-5c0f-4f09-b9df-9f127d02cb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:34.891886 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c88c53-5c0f-4f09-b9df-9f127d02cb1d admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:34.892165 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c88c53-5c0f-4f09-b9df-9f127d02cb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:34.892437 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c88c53-5c0f-4f09-b9df-9f127d02cb1d admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:34.893077 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c88c53-5c0f-4f09-b9df-9f127d02cb1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:34.895041 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c88c53-5c0f-4f09-b9df-9f127d02cb1d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:34.994039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1405/2770] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:04:34 2022] POST /identity/v3/users => generated 291 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:04:34.996599 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165548d4-4db6-40b3-8cca-10ba9f42de1b admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:35.008412 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165548d4-4db6-40b3-8cca-10ba9f42de1b admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:35.014381 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3783-438a-4f6a-982f-ad393fc92974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:35.015062 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3783-438a-4f6a-982f-ad393fc92974 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:35.015345 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3783-438a-4f6a-982f-ad393fc92974 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:35.015622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3783-438a-4f6a-982f-ad393fc92974 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:35.016273 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd3783-438a-4f6a-982f-ad393fc92974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:35.017885 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd3783-438a-4f6a-982f-ad393fc92974 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:35.030531 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1366/2771] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:04:35.034736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c88c53-5c0f-4f09-b9df-9f127d02cb1d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:35.045243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c88c53-5c0f-4f09-b9df-9f127d02cb1d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:35.051302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa7d44-4c00-4417-9447-1525b58f4507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:35.052141 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa7d44-4c00-4417-9447-1525b58f4507 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:35.052448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa7d44-4c00-4417-9447-1525b58f4507 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:35.052855 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa7d44-4c00-4417-9447-1525b58f4507 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:35.053518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa7d44-4c00-4417-9447-1525b58f4507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:35.055176 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa7d44-4c00-4417-9447-1525b58f4507 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:35.066825 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1406/2772] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:04:35.070887 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3783-438a-4f6a-982f-ad393fc92974 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:35.080911 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3783-438a-4f6a-982f-ad393fc92974 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:35.085733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917bee8-1333-4781-8a61-48a6b9369b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:35.086375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917bee8-1333-4781-8a61-48a6b9369b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:35.086687 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917bee8-1333-4781-8a61-48a6b9369b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:35.086963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917bee8-1333-4781-8a61-48a6b9369b8b admin admin] PATH_INFO: `/v3/projects/372ca5d37ce94b8fa267f2ee7a81789e/users/c9ac7bf809174b6680a4eb5e088de614/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:35.102304 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917bee8-1333-4781-8a61-48a6b9369b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=372ca5d37ce94b8fa267f2ee7a81789e, user_id=c9ac7bf809174b6680a4eb5e088de614, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:35.105643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917bee8-1333-4781-8a61-48a6b9369b8b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:35.162021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1367/2773] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:04:35 2022] PUT /identity/v3/projects/372ca5d37ce94b8fa267f2ee7a81789e/users/c9ac7bf809174b6680a4eb5e088de614/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:04:35.167535 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786ffb1-1036-4605-95d8-c08b3132d831 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:35.167819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786ffb1-1036-4605-95d8-c08b3132d831 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:35.168083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786ffb1-1036-4605-95d8-c08b3132d831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:35.187382 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-7786ffb1-1036-4605-95d8-c08b3132d831 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:35.188562 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7786ffb1-1036-4605-95d8-c08b3132d831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:35.246405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1407/2774] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:35.256134 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917bee8-1333-4781-8a61-48a6b9369b8b admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:35.257921 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3917bee8-1333-4781-8a61-48a6b9369b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:35.333446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917bee8-1333-4781-8a61-48a6b9369b8b admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:35.338710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345632a2-461a-4e82-9e44-8116001a58db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:35.339348 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345632a2-461a-4e82-9e44-8116001a58db service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:35.339740 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345632a2-461a-4e82-9e44-8116001a58db service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:35.340123 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345632a2-461a-4e82-9e44-8116001a58db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:35.350527 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345632a2-461a-4e82-9e44-8116001a58db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:35.352043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345632a2-461a-4e82-9e44-8116001a58db service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:35.364234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1368/2775] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:04:36.488960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f4336-6cce-49ee-ac2f-c65317cd075e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:36.489482 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f4336-6cce-49ee-ac2f-c65317cd075e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:36.489482 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f4336-6cce-49ee-ac2f-c65317cd075e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:36.498833 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:36.525765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-1d2f4336-6cce-49ee-ac2f-c65317cd075e None None] MFA Rules not processed for user `c9ac7bf809174b6680a4eb5e088de614`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:36.527842 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1d2f4336-6cce-49ee-ac2f-c65317cd075e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:36.598233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1408/2776] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:36.609747 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345632a2-461a-4e82-9e44-8116001a58db service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:36.611197 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-345632a2-461a-4e82-9e44-8116001a58db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:36.668814 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345632a2-461a-4e82-9e44-8116001a58db service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:36.672438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc75fb1-7a67-4dca-bd5b-b3c453091c90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:36.672982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc75fb1-7a67-4dca-bd5b-b3c453091c90 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:36.673386 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc75fb1-7a67-4dca-bd5b-b3c453091c90 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:36.673675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc75fb1-7a67-4dca-bd5b-b3c453091c90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:36.682255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc75fb1-7a67-4dca-bd5b-b3c453091c90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:36.684307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc75fb1-7a67-4dca-bd5b-b3c453091c90 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:36.695805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1369/2777] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:04:38.346725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37099de8-f641-48f0-b058-8da1231875bf None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:38.347261 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37099de8-f641-48f0-b058-8da1231875bf None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:38.347261 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37099de8-f641-48f0-b058-8da1231875bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:38.368007 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-37099de8-f641-48f0-b058-8da1231875bf None None] MFA Rules not processed for user `c9ac7bf809174b6680a4eb5e088de614`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:04:38.369217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-37099de8-f641-48f0-b058-8da1231875bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:38.376836 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1409/2778] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:04:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:04:40.359199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc75fb1-7a67-4dca-bd5b-b3c453091c90 service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:40.360662 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3cc75fb1-7a67-4dca-bd5b-b3c453091c90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:40.379875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:40.462532 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc75fb1-7a67-4dca-bd5b-b3c453091c90 service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:40.467746 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acbc3b-f783-490f-a880-01b72648bb55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:40.468352 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acbc3b-f783-490f-a880-01b72648bb55 service placement] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:40.468628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acbc3b-f783-490f-a880-01b72648bb55 service placement] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:40.468897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acbc3b-f783-490f-a880-01b72648bb55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:40.470689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-72acbc3b-f783-490f-a880-01b72648bb55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:40.482925 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acbc3b-f783-490f-a880-01b72648bb55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:40.484531 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acbc3b-f783-490f-a880-01b72648bb55 service placement] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:40.494732 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1370/2779] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:04:48.361501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=33 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:51.024399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=34 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:04:54.021361 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37099de8-f641-48f0-b058-8da1231875bf None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:04:54.023048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-37099de8-f641-48f0-b058-8da1231875bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:54.035905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37099de8-f641-48f0-b058-8da1231875bf None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:04:54.039725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3964e-8674-484c-b0b6-71bd1fb7b32b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:04:54.040349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3964e-8674-484c-b0b6-71bd1fb7b32b service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:04:54.040694 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3964e-8674-484c-b0b6-71bd1fb7b32b service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:04:54.041139 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3964e-8674-484c-b0b6-71bd1fb7b32b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:04:54.042858 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1da3964e-8674-484c-b0b6-71bd1fb7b32b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:04:54.113489 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3964e-8674-484c-b0b6-71bd1fb7b32b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:04:54.114983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3964e-8674-484c-b0b6-71bd1fb7b32b service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:04:54.126649 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1410/2780] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:04:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:05:04.120196 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=30 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:36.280338 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acbc3b-f783-490f-a880-01b72648bb55 service placement] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:36.291313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acbc3b-f783-490f-a880-01b72648bb55 service placement] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:36.295069 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a8d80-5e60-4061-a551-e68145a7579f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:36.295664 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a8d80-5e60-4061-a551-e68145a7579f service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:36.295941 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a8d80-5e60-4061-a551-e68145a7579f service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:36.296210 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a8d80-5e60-4061-a551-e68145a7579f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:36.298044 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a54a8d80-5e60-4061-a551-e68145a7579f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:05:36.308104 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a8d80-5e60-4061-a551-e68145a7579f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:36.309596 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a8d80-5e60-4061-a551-e68145a7579f service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:36.317712 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1371/2781] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:05:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:05:36.395344 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3964e-8674-484c-b0b6-71bd1fb7b32b service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:05:36.404517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3964e-8674-484c-b0b6-71bd1fb7b32b service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:05:36.408034 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afe537-a984-4201-8807-628d76076f9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:05:36.408624 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afe537-a984-4201-8807-628d76076f9d service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:36.408897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afe537-a984-4201-8807-628d76076f9d service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:36.409163 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afe537-a984-4201-8807-628d76076f9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:36.410864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-65afe537-a984-4201-8807-628d76076f9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:05:36.491985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afe537-a984-4201-8807-628d76076f9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:05:36.493461 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afe537-a984-4201-8807-628d76076f9d service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:05:36.514150 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1411/2782] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:05:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:05:46.314049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=6 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:46.511524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=43 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:05:50.813060 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24a8bc-6e02-4191-b60c-370582b0c41e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:05:50.813539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24a8bc-6e02-4191-b60c-370582b0c41e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:05:50.813655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24a8bc-6e02-4191-b60c-370582b0c41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:05:50.832194 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-3c24a8bc-6e02-4191-b60c-370582b0c41e None None] MFA Rules not processed for user `c9ac7bf809174b6680a4eb5e088de614`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:05:50.833628 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-3c24a8bc-6e02-4191-b60c-370582b0c41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:05:50.840810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1372/2783] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:05:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:06:00.827313 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:25.904855 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afe537-a984-4201-8807-628d76076f9d service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:06:25.914066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afe537-a984-4201-8807-628d76076f9d service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:06:25.917877 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4c7e8-4e55-4ddd-ac30-831127e59492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:06:25.918467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4c7e8-4e55-4ddd-ac30-831127e59492 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:25.918774 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4c7e8-4e55-4ddd-ac30-831127e59492 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:25.919043 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4c7e8-4e55-4ddd-ac30-831127e59492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:25.920824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-8cb4c7e8-4e55-4ddd-ac30-831127e59492 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:25.934546 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4c7e8-4e55-4ddd-ac30-831127e59492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:06:25.939512 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4c7e8-4e55-4ddd-ac30-831127e59492 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:06:25.946792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1412/2784] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:06:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:06:28.840519 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24a8bc-6e02-4191-b60c-370582b0c41e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:06:28.850122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24a8bc-6e02-4191-b60c-370582b0c41e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:06:28.854214 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf6b1d-a38c-4436-b67e-3dca8b5f52e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:06:28.854840 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf6b1d-a38c-4436-b67e-3dca8b5f52e7 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:28.855115 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf6b1d-a38c-4436-b67e-3dca8b5f52e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:28.855384 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf6b1d-a38c-4436-b67e-3dca8b5f52e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:28.857137 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-03cf6b1d-a38c-4436-b67e-3dca8b5f52e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:28.871017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf6b1d-a38c-4436-b67e-3dca8b5f52e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:06:28.872425 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf6b1d-a38c-4436-b67e-3dca8b5f52e7 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:06:28.883202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1373/2785] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:06:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:06:35.942014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=6 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:38.878339 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=6 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:06:57.767187 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4c7e8-4e55-4ddd-ac30-831127e59492 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:06:57.781348 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4c7e8-4e55-4ddd-ac30-831127e59492 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:06:57.784370 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:06:57.784972 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:57.785244 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:57.785511 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] PATH_INFO: `/v3/users/c9ac7bf809174b6680a4eb5e088de614` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:57.786763 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9ac7bf809174b6680a4eb5e088de614)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:06:57.788827 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:06:57.980162 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9ac7bf809174b6680a4eb5e088de614'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:06:58.062986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9ac7bf809174b6680a4eb5e088de614'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:06:58.063721 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9ac7bf809174b6680a4eb5e088de614'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:06:58.064573 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9ac7bf809174b6680a4eb5e088de614'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:06:58.079667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9ac7bf809174b6680a4eb5e088de614'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:06:58.094505 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1413/2786] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:06:57 2022] DELETE /identity/v3/users/c9ac7bf809174b6680a4eb5e088de614 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:06:58.466666 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf6b1d-a38c-4436-b67e-3dca8b5f52e7 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:06:58.480902 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf6b1d-a38c-4436-b67e-3dca8b5f52e7 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:06:58.544069 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246253da-8e2b-4a15-830f-8029cf9c75f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:06:58.544810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246253da-8e2b-4a15-830f-8029cf9c75f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:58.545185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246253da-8e2b-4a15-830f-8029cf9c75f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:58.545380 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246253da-8e2b-4a15-830f-8029cf9c75f0 admin admin] PATH_INFO: `/v3/projects/372ca5d37ce94b8fa267f2ee7a81789e` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:58.546563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246253da-8e2b-4a15-830f-8029cf9c75f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=372ca5d37ce94b8fa267f2ee7a81789e)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:06:58.548477 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246253da-8e2b-4a15-830f-8029cf9c75f0 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:06:58.664864 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-246253da-8e2b-4a15-830f-8029cf9c75f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '372ca5d37ce94b8fa267f2ee7a81789e', 'request_id': 'req-246253da-8e2b-4a15-830f-8029cf9c75f0'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:06:58.672683 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-246253da-8e2b-4a15-830f-8029cf9c75f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '372ca5d37ce94b8fa267f2ee7a81789e', 'request_id': 'req-246253da-8e2b-4a15-830f-8029cf9c75f0'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:06:58.743622 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-246253da-8e2b-4a15-830f-8029cf9c75f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '372ca5d37ce94b8fa267f2ee7a81789e', 'request_id': 'req-246253da-8e2b-4a15-830f-8029cf9c75f0'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:06:58.770773 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-246253da-8e2b-4a15-830f-8029cf9c75f0 admin admin] The token cache is being invalidate because project 372ca5d37ce94b8fa267f2ee7a81789e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:06:58.775830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-246253da-8e2b-4a15-830f-8029cf9c75f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:06:58.775830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-246253da-8e2b-4a15-830f-8029cf9c75f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:06:58.776973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1374/2787] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:06:58 2022] DELETE /identity/v3/projects/372ca5d37ce94b8fa267f2ee7a81789e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:06:59.264114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:06:59.268110 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:59.337124 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1aec9-2a13-4cdb-87f6-0026f628b720 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:06:59.409519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da814f5-cc8e-4e0a-9dd6-c5076c9af602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:06:59.410140 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da814f5-cc8e-4e0a-9dd6-c5076c9af602 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:59.410419 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da814f5-cc8e-4e0a-9dd6-c5076c9af602 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:59.410728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da814f5-cc8e-4e0a-9dd6-c5076c9af602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:59.413233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2da814f5-cc8e-4e0a-9dd6-c5076c9af602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:59.431555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da814f5-cc8e-4e0a-9dd6-c5076c9af602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:06:59.431555 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da814f5-cc8e-4e0a-9dd6-c5076c9af602 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:06:59.437399 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1414/2788] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:06:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:06:59.755203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7e1d6-6b40-43c1-b4ed-febd08d7f2b5 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:06:59.756159 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7e1d6-6b40-43c1-b4ed-febd08d7f2b5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:06:59.756498 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7e1d6-6b40-43c1-b4ed-febd08d7f2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:06:59.779225 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-78c7e1d6-6b40-43c1-b4ed-febd08d7f2b5 None None] MFA Rules not processed for user `bc2cc611fd9341fc9a6d8a5d49f43e53`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:06:59.780973 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-78c7e1d6-6b40-43c1-b4ed-febd08d7f2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:06:59.841287 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1375/2789] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:06:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:07:01.640547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da814f5-cc8e-4e0a-9dd6-c5076c9af602 service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:01.644418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-2da814f5-cc8e-4e0a-9dd6-c5076c9af602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:01.683963 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:01.729408 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da814f5-cc8e-4e0a-9dd6-c5076c9af602 service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:01.735341 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe16115-7ca4-415e-bcef-188c38ad3401 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:01.736001 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe16115-7ca4-415e-bcef-188c38ad3401 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:01.736266 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe16115-7ca4-415e-bcef-188c38ad3401 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:01.736829 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe16115-7ca4-415e-bcef-188c38ad3401 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:01.738318 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ebe16115-7ca4-415e-bcef-188c38ad3401 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:01.752784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe16115-7ca4-415e-bcef-188c38ad3401 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:01.754387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe16115-7ca4-415e-bcef-188c38ad3401 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:01.767123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1415/2790] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:06.555305 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7e1d6-6b40-43c1-b4ed-febd08d7f2b5 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:06.556740 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-78c7e1d6-6b40-43c1-b4ed-febd08d7f2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:06.569513 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7e1d6-6b40-43c1-b4ed-febd08d7f2b5 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:06.573101 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dae3b7-f207-4b8f-b867-68c6f9fc892c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:06.573702 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dae3b7-f207-4b8f-b867-68c6f9fc892c service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:06.573978 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dae3b7-f207-4b8f-b867-68c6f9fc892c service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:06.574251 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dae3b7-f207-4b8f-b867-68c6f9fc892c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:06.575995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d8dae3b7-f207-4b8f-b867-68c6f9fc892c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:06.610750 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:06.641836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dae3b7-f207-4b8f-b867-68c6f9fc892c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:06.643311 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dae3b7-f207-4b8f-b867-68c6f9fc892c service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:06.654141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1376/2791] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:11.901909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=20 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:14.196320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe16115-7ca4-415e-bcef-188c38ad3401 service nova] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:14.198925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-ebe16115-7ca4-415e-bcef-188c38ad3401 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:14.213969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe16115-7ca4-415e-bcef-188c38ad3401 service nova] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:14.218969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:14.219705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:14.220103 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:14.220380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] PATH_INFO: `/v3/users/4c96322c48c146c7b8bcb9959f533820` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:14.221504 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c96322c48c146c7b8bcb9959f533820)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:14.223440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:14.392455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c96322c48c146c7b8bcb9959f533820'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:14.449675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c96322c48c146c7b8bcb9959f533820'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:14.450321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c96322c48c146c7b8bcb9959f533820'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:14.450950 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c96322c48c146c7b8bcb9959f533820'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:14.464274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c96322c48c146c7b8bcb9959f533820'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:14.476847 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1416/2792] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:07:14 2022] DELETE /identity/v3/users/4c96322c48c146c7b8bcb9959f533820 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:07:14.723257 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dae3b7-f207-4b8f-b867-68c6f9fc892c service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:14.732364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dae3b7-f207-4b8f-b867-68c6f9fc892c service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:14.790315 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:14.791741 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:14.792278 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:14.792786 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] PATH_INFO: `/v3/projects/d427be7da6084e009b87ff500858cdc8` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:14.793909 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d427be7da6084e009b87ff500858cdc8)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:14.795854 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:14.903896 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd427be7da6084e009b87ff500858cdc8', 'request_id': 'req-93327d33-c0fc-4e78-a0ee-d3f443eef98a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:14.910335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd427be7da6084e009b87ff500858cdc8', 'request_id': 'req-93327d33-c0fc-4e78-a0ee-d3f443eef98a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:14.981239 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd427be7da6084e009b87ff500858cdc8', 'request_id': 'req-93327d33-c0fc-4e78-a0ee-d3f443eef98a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:14.992254 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] The token cache is being invalidate because project d427be7da6084e009b87ff500858cdc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:07:14.992538 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:14.993181 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:14.995527 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1377/2793] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:07:14 2022] DELETE /identity/v3/projects/d427be7da6084e009b87ff500858cdc8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:07:14.999585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:15.001083 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:15.015394 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924c2b-9a8a-4a0e-9a49-1344f421c449 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:15.084440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:15.085208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:15.085495 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:15.085767 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] PATH_INFO: `/v3/users/fe7cff0184954f0d93e9aa9b5c346c75` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:15.086955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe7cff0184954f0d93e9aa9b5c346c75)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:15.088962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:15.272448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe7cff0184954f0d93e9aa9b5c346c75'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:15.337533 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe7cff0184954f0d93e9aa9b5c346c75'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:15.339147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe7cff0184954f0d93e9aa9b5c346c75'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:15.340048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe7cff0184954f0d93e9aa9b5c346c75'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:15.354430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe7cff0184954f0d93e9aa9b5c346c75'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:15.358598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:15.379322 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1417/2794] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:07:14 2022] DELETE /identity/v3/users/fe7cff0184954f0d93e9aa9b5c346c75 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:07:15.691938 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:15.693270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:15.712707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93327d33-c0fc-4e78-a0ee-d3f443eef98a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:15.778621 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:15.779293 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:15.779528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:15.779850 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] PATH_INFO: `/v3/projects/065af61a31cd48ddb0299de7aad02a15` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:15.781031 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=065af61a31cd48ddb0299de7aad02a15)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:15.782974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:15.784701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:15.879868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065af61a31cd48ddb0299de7aad02a15', 'request_id': 'req-83bd2516-52cf-440f-852a-613fed839ebf'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:15.885928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065af61a31cd48ddb0299de7aad02a15', 'request_id': 'req-83bd2516-52cf-440f-852a-613fed839ebf'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:15.937416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065af61a31cd48ddb0299de7aad02a15', 'request_id': 'req-83bd2516-52cf-440f-852a-613fed839ebf'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:15.948576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] The token cache is being invalidate because project 065af61a31cd48ddb0299de7aad02a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:07:15.948868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:15.949508 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:15.951034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1378/2795] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:07:15 2022] DELETE /identity/v3/projects/065af61a31cd48ddb0299de7aad02a15 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:07:25.593444 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:25.960982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=24,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:31.702235 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:31.703601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:31.775791 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75efd1-a05f-4671-968c-fcfe06f77b9d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:31.837584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180c82c-b549-4454-9185-b9cd1b7263b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:31.838190 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180c82c-b549-4454-9185-b9cd1b7263b8 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:31.838468 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180c82c-b549-4454-9185-b9cd1b7263b8 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:31.838775 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180c82c-b549-4454-9185-b9cd1b7263b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:31.841114 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c180c82c-b549-4454-9185-b9cd1b7263b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:31.902024 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180c82c-b549-4454-9185-b9cd1b7263b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:31.904485 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180c82c-b549-4454-9185-b9cd1b7263b8 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:31.913302 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1418/2796] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:32.295073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:32.296613 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:32.356512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd2516-52cf-440f-852a-613fed839ebf admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:32.360199 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e01056-f355-443f-9f15-58a0e6a7249b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:32.360810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e01056-f355-443f-9f15-58a0e6a7249b service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:32.361091 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e01056-f355-443f-9f15-58a0e6a7249b service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:32.361364 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e01056-f355-443f-9f15-58a0e6a7249b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:32.363121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-37e01056-f355-443f-9f15-58a0e6a7249b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:32.425824 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e01056-f355-443f-9f15-58a0e6a7249b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:32.427270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e01056-f355-443f-9f15-58a0e6a7249b service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:32.436849 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1379/2797] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:36.049012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180c82c-b549-4454-9185-b9cd1b7263b8 service nova] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:36.050466 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c180c82c-b549-4454-9185-b9cd1b7263b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:36.094123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:36.121723 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180c82c-b549-4454-9185-b9cd1b7263b8 service nova] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:36.127198 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72a1c3-f3ee-46e9-9487-d9283bb3db1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:36.132290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72a1c3-f3ee-46e9-9487-d9283bb3db1e service placement] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:36.132290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72a1c3-f3ee-46e9-9487-d9283bb3db1e service placement] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:36.132290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72a1c3-f3ee-46e9-9487-d9283bb3db1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:36.132290 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-4f72a1c3-f3ee-46e9-9487-d9283bb3db1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:36.141716 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f72a1c3-f3ee-46e9-9487-d9283bb3db1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:36.143303 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f72a1c3-f3ee-46e9-9487-d9283bb3db1e service placement] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:36.153853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1419/2798] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:07:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:07:39.596144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e01056-f355-443f-9f15-58a0e6a7249b service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:39.600192 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-37e01056-f355-443f-9f15-58a0e6a7249b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:39.611144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e01056-f355-443f-9f15-58a0e6a7249b service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:39.615901 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:39.616535 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:39.616816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:39.617088 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] PATH_INFO: `/v3/users/a112ff54558147cf8739322b085fe1ab` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:39.618307 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a112ff54558147cf8739322b085fe1ab)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:39.620177 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:39.822626 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a112ff54558147cf8739322b085fe1ab'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:39.891565 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a112ff54558147cf8739322b085fe1ab'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:39.892152 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a112ff54558147cf8739322b085fe1ab'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:39.892765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a112ff54558147cf8739322b085fe1ab'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:39.900255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a112ff54558147cf8739322b085fe1ab'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:39.918790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1380/2799] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:07:39 2022] DELETE /identity/v3/users/a112ff54558147cf8739322b085fe1ab => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:07:40.184931 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72a1c3-f3ee-46e9-9487-d9283bb3db1e service placement] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:40.196661 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72a1c3-f3ee-46e9-9487-d9283bb3db1e service placement] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:40.254900 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:40.256496 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:40.256777 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:40.257050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] PATH_INFO: `/v3/projects/add9aa988a0848d185d4f06039cfd00e` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:40.258156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=add9aa988a0848d185d4f06039cfd00e)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:40.260079 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:40.354059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'add9aa988a0848d185d4f06039cfd00e', 'request_id': 'req-768f00b0-5083-4ee1-8dfa-c67645422ff8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:40.360397 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'add9aa988a0848d185d4f06039cfd00e', 'request_id': 'req-768f00b0-5083-4ee1-8dfa-c67645422ff8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:40.439604 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'add9aa988a0848d185d4f06039cfd00e', 'request_id': 'req-768f00b0-5083-4ee1-8dfa-c67645422ff8'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:40.450913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] The token cache is being invalidate because project add9aa988a0848d185d4f06039cfd00e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:07:40.451202 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:40.451826 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:40.453309 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1420/2800] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:07:40 2022] DELETE /identity/v3/projects/add9aa988a0848d185d4f06039cfd00e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:07:40.457597 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:40.459073 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:40.473564 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd0cdd-fabd-4f2e-9445-5ca08fc40d19 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:40.491789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:40.531974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:40.532600 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:40.532881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:40.533154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] PATH_INFO: `/v3/users/e8c4e93409dd40ad85db4f6c5bfe8e9a` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:40.534397 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c4e93409dd40ad85db4f6c5bfe8e9a)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:40.536333 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:40.765610 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c4e93409dd40ad85db4f6c5bfe8e9a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:40.839439 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c4e93409dd40ad85db4f6c5bfe8e9a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:40.840087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c4e93409dd40ad85db4f6c5bfe8e9a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:40.840983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c4e93409dd40ad85db4f6c5bfe8e9a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:40.848279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c4e93409dd40ad85db4f6c5bfe8e9a'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:40.865975 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1381/2801] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:07:40 2022] DELETE /identity/v3/users/e8c4e93409dd40ad85db4f6c5bfe8e9a => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:07:41.130524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:07:41.131938 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:07:41.145009 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f00b0-5083-4ee1-8dfa-c67645422ff8 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:07:41.195637 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:41.202340 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:07:41.203107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:07:41.203421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:07:41.206367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] PATH_INFO: `/v3/projects/955441331aa64ee7b01ffbbb7016915c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:07:41.206367 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=955441331aa64ee7b01ffbbb7016915c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:07:41.206874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:07:41.325743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '955441331aa64ee7b01ffbbb7016915c', 'request_id': 'req-c16b8684-41b7-4bfb-bc87-78bbb8644d61'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:41.332072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '955441331aa64ee7b01ffbbb7016915c', 'request_id': 'req-c16b8684-41b7-4bfb-bc87-78bbb8644d61'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:41.397057 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '955441331aa64ee7b01ffbbb7016915c', 'request_id': 'req-c16b8684-41b7-4bfb-bc87-78bbb8644d61'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:41.408332 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] The token cache is being invalidate because project 955441331aa64ee7b01ffbbb7016915c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:07:41.408619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:41.409243 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:07:41.410765 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1421/2802] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:07:41 2022] DELETE /identity/v3/projects/955441331aa64ee7b01ffbbb7016915c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:07:50.862703 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:07:51.405208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:22.415609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:22.416992 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:22.473855 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9104828-cc6f-49f8-8a2a-c69c41c6a590 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:22.520647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddbe4e-aa84-4847-af1f-ef680e4986ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:22.521244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddbe4e-aa84-4847-af1f-ef680e4986ec service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:22.521517 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddbe4e-aa84-4847-af1f-ef680e4986ec service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:22.521784 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddbe4e-aa84-4847-af1f-ef680e4986ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:22.523499 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e3ddbe4e-aa84-4847-af1f-ef680e4986ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:22.587982 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ddbe4e-aa84-4847-af1f-ef680e4986ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:22.589411 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ddbe4e-aa84-4847-af1f-ef680e4986ec service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:22.600024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1382/2803] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:26.084650 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:26.086915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:26.160959 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b8684-41b7-4bfb-bc87-78bbb8644d61 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:26.166004 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c67a2-9c90-476a-8a41-30cf99e5dfed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:26.166639 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c67a2-9c90-476a-8a41-30cf99e5dfed service placement] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:26.166916 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c67a2-9c90-476a-8a41-30cf99e5dfed service placement] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:26.167184 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c67a2-9c90-476a-8a41-30cf99e5dfed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:26.168928 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b49c67a2-9c90-476a-8a41-30cf99e5dfed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:26.180843 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c67a2-9c90-476a-8a41-30cf99e5dfed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:26.182380 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c67a2-9c90-476a-8a41-30cf99e5dfed service placement] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:26.192465 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1422/2804] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:26.207034 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddbe4e-aa84-4847-af1f-ef680e4986ec service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:26.208583 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e3ddbe4e-aa84-4847-af1f-ef680e4986ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:26.221788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddbe4e-aa84-4847-af1f-ef680e4986ec service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:26.230898 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77027f86-bec8-4042-aa82-d71135f624ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:26.231550 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77027f86-bec8-4042-aa82-d71135f624ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:26.231830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77027f86-bec8-4042-aa82-d71135f624ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:26.232106 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77027f86-bec8-4042-aa82-d71135f624ac admin admin] PATH_INFO: `/v3/users/bc2cc611fd9341fc9a6d8a5d49f43e53` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:26.233268 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77027f86-bec8-4042-aa82-d71135f624ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc2cc611fd9341fc9a6d8a5d49f43e53)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:26.235206 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77027f86-bec8-4042-aa82-d71135f624ac admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:26.403886 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-77027f86-bec8-4042-aa82-d71135f624ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc2cc611fd9341fc9a6d8a5d49f43e53'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:08:26.463331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-77027f86-bec8-4042-aa82-d71135f624ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc2cc611fd9341fc9a6d8a5d49f43e53'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:08:26.463867 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-77027f86-bec8-4042-aa82-d71135f624ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc2cc611fd9341fc9a6d8a5d49f43e53'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:08:26.466121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-77027f86-bec8-4042-aa82-d71135f624ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc2cc611fd9341fc9a6d8a5d49f43e53'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:08:26.475061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:26.477409 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-77027f86-bec8-4042-aa82-d71135f624ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc2cc611fd9341fc9a6d8a5d49f43e53'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:08:26.501681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1383/2805] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:08:26 2022] DELETE /identity/v3/users/bc2cc611fd9341fc9a6d8a5d49f43e53 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:08:26.739981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c67a2-9c90-476a-8a41-30cf99e5dfed service placement] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:26.751548 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c67a2-9c90-476a-8a41-30cf99e5dfed service placement] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:26.814656 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:26.816060 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:26.816466 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:26.816870 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78 admin admin] PATH_INFO: `/v3/projects/31f8b13a6a4844fc9fd20ab2c3ff7368` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:26.818463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31f8b13a6a4844fc9fd20ab2c3ff7368)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:26.821442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:26.925725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31f8b13a6a4844fc9fd20ab2c3ff7368', 'request_id': 'req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:08:26.931791 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31f8b13a6a4844fc9fd20ab2c3ff7368', 'request_id': 'req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:08:27.004660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31f8b13a6a4844fc9fd20ab2c3ff7368', 'request_id': 'req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:08:27.016233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78 admin admin] The token cache is being invalidate because project 31f8b13a6a4844fc9fd20ab2c3ff7368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:08:27.016517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:08:27.017177 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-5bed06a9-ad6c-4e28-bf04-8639c4f87a78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:08:27.018898 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1423/2806] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:08:26 2022] DELETE /identity/v3/projects/31f8b13a6a4844fc9fd20ab2c3ff7368 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:08:27.024693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e69a81-ce97-4c0b-b9f6-f5c7a28a8083 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:27.083122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e69a81-ce97-4c0b-b9f6-f5c7a28a8083 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:27.083122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e69a81-ce97-4c0b-b9f6-f5c7a28a8083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:27.083122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-98e69a81-ce97-4c0b-b9f6-f5c7a28a8083 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:27.083122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-98e69a81-ce97-4c0b-b9f6-f5c7a28a8083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:27.107370 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1384/2807] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 13:08:27.115735 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5b87f-3a64-4a93-b5ad-f1160a09e6f3 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:27.124388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5b87f-3a64-4a93-b5ad-f1160a09e6f3 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:27.124388 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5b87f-3a64-4a93-b5ad-f1160a09e6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:27.136529 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-a8b5b87f-3a64-4a93-b5ad-f1160a09e6f3 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:27.137778 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-a8b5b87f-3a64-4a93-b5ad-f1160a09e6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:27.145837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1424/2808] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:27.150101 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e69a81-ce97-4c0b-b9f6-f5c7a28a8083 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:27.159931 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e69a81-ce97-4c0b-b9f6-f5c7a28a8083 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:27.164516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fadc9-48a3-41e7-8a55-b40fa2ac0054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:27.165111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fadc9-48a3-41e7-8a55-b40fa2ac0054 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:27.165393 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fadc9-48a3-41e7-8a55-b40fa2ac0054 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:27.165671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fadc9-48a3-41e7-8a55-b40fa2ac0054 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:27.166371 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fadc9-48a3-41e7-8a55-b40fa2ac0054 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:08:27.166780 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fadc9-48a3-41e7-8a55-b40fa2ac0054 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:27.168357 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fadc9-48a3-41e7-8a55-b40fa2ac0054 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:27.180426 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1385/2809] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:08:27.190992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5b87f-3a64-4a93-b5ad-f1160a09e6f3 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:27.200113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5b87f-3a64-4a93-b5ad-f1160a09e6f3 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:27.204743 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917838c0-e794-424c-9bd4-50c62687fabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:27.205333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917838c0-e794-424c-9bd4-50c62687fabe admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:27.205651 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917838c0-e794-424c-9bd4-50c62687fabe admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:27.205922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917838c0-e794-424c-9bd4-50c62687fabe admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:27.206588 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917838c0-e794-424c-9bd4-50c62687fabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:27.210149 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917838c0-e794-424c-9bd4-50c62687fabe admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:27.220040 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:27.309132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1425/2810] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:08:27 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:08:27.313089 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fadc9-48a3-41e7-8a55-b40fa2ac0054 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:27.324407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fadc9-48a3-41e7-8a55-b40fa2ac0054 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:27.381771 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58f2bd-2800-49b1-831b-1e2f4678ba4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:27.382368 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58f2bd-2800-49b1-831b-1e2f4678ba4b admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:27.382683 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58f2bd-2800-49b1-831b-1e2f4678ba4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:27.382953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58f2bd-2800-49b1-831b-1e2f4678ba4b admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:27.383566 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58f2bd-2800-49b1-831b-1e2f4678ba4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:27.385439 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58f2bd-2800-49b1-831b-1e2f4678ba4b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:27.463950 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1386/2811] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:08:27 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:08:27.467978 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917838c0-e794-424c-9bd4-50c62687fabe admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:27.479208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917838c0-e794-424c-9bd4-50c62687fabe admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:27.484260 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3e104-0d67-42be-a7f5-ff19a3a60309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:27.484924 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3e104-0d67-42be-a7f5-ff19a3a60309 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:27.485427 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3e104-0d67-42be-a7f5-ff19a3a60309 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:27.485660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3e104-0d67-42be-a7f5-ff19a3a60309 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:27.486156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3e104-0d67-42be-a7f5-ff19a3a60309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:27.488971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3e104-0d67-42be-a7f5-ff19a3a60309 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:27.499396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1426/2812] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:08:27.503448 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58f2bd-2800-49b1-831b-1e2f4678ba4b admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:27.513671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58f2bd-2800-49b1-831b-1e2f4678ba4b admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:27.518383 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6641f86-59eb-40f9-bc17-d397b0086d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:27.519012 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6641f86-59eb-40f9-bc17-d397b0086d8b admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:27.519295 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6641f86-59eb-40f9-bc17-d397b0086d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:27.519570 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6641f86-59eb-40f9-bc17-d397b0086d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:27.520200 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6641f86-59eb-40f9-bc17-d397b0086d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:27.521782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6641f86-59eb-40f9-bc17-d397b0086d8b admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:27.532191 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1387/2813] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:08:27.537002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3e104-0d67-42be-a7f5-ff19a3a60309 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:27.546849 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3e104-0d67-42be-a7f5-ff19a3a60309 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:27.552336 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64466cab-4660-47cf-a060-0e5edb462189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:27.552970 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64466cab-4660-47cf-a060-0e5edb462189 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:27.553248 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64466cab-4660-47cf-a060-0e5edb462189 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:27.553520 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64466cab-4660-47cf-a060-0e5edb462189 admin admin] PATH_INFO: `/v3/projects/63dc094692fd489ea9d3548f2b555ed7/users/8b76119bd626453a859c8716605d049c/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:27.567953 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64466cab-4660-47cf-a060-0e5edb462189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63dc094692fd489ea9d3548f2b555ed7, user_id=8b76119bd626453a859c8716605d049c, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:27.570622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64466cab-4660-47cf-a060-0e5edb462189 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:27.648476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1427/2814] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:08:27 2022] PUT /identity/v3/projects/63dc094692fd489ea9d3548f2b555ed7/users/8b76119bd626453a859c8716605d049c/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:08:27.660128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47c64d-91e1-45c4-a2b8-014a74c63d42 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:27.660442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47c64d-91e1-45c4-a2b8-014a74c63d42 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:27.660717 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47c64d-91e1-45c4-a2b8-014a74c63d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:27.687759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-9b47c64d-91e1-45c4-a2b8-014a74c63d42 None None] MFA Rules not processed for user `8b76119bd626453a859c8716605d049c`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:27.688998 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-9b47c64d-91e1-45c4-a2b8-014a74c63d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:27.721724 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:27.753746 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1388/2815] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:08:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:27.758339 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64466cab-4660-47cf-a060-0e5edb462189 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:27.820196 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64466cab-4660-47cf-a060-0e5edb462189 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:27.824913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5dfbe-9317-45cf-95b5-fca8b54f52a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:27.827106 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5dfbe-9317-45cf-95b5-fca8b54f52a5 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:27.827106 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5dfbe-9317-45cf-95b5-fca8b54f52a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:27.827106 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5dfbe-9317-45cf-95b5-fca8b54f52a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:27.827106 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5dfbe-9317-45cf-95b5-fca8b54f52a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:27.828687 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5dfbe-9317-45cf-95b5-fca8b54f52a5 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:27.945855 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1428/2816] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:08:27 2022] POST /identity/v3/projects => generated 371 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:08:27.949934 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47c64d-91e1-45c4-a2b8-014a74c63d42 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:27.961675 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47c64d-91e1-45c4-a2b8-014a74c63d42 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:28.020267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c00cc3-f735-49ce-b1ae-1950a3c013b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:28.020868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c00cc3-f735-49ce-b1ae-1950a3c013b2 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:28.021149 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c00cc3-f735-49ce-b1ae-1950a3c013b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:28.021423 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c00cc3-f735-49ce-b1ae-1950a3c013b2 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:28.022042 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c00cc3-f735-49ce-b1ae-1950a3c013b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:28.024090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c00cc3-f735-49ce-b1ae-1950a3c013b2 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:28.096255 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1389/2817] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:08:27 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:08:28.100752 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5dfbe-9317-45cf-95b5-fca8b54f52a5 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:28.111436 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5dfbe-9317-45cf-95b5-fca8b54f52a5 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:28.116686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c491f3c-94f1-41c3-8ce3-f67d6ca05124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:28.117286 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c491f3c-94f1-41c3-8ce3-f67d6ca05124 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:28.117549 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c491f3c-94f1-41c3-8ce3-f67d6ca05124 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:28.117819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c491f3c-94f1-41c3-8ce3-f67d6ca05124 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:28.118462 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c491f3c-94f1-41c3-8ce3-f67d6ca05124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:28.120096 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c491f3c-94f1-41c3-8ce3-f67d6ca05124 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:28.132532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1429/2818] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:08:28.136782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c00cc3-f735-49ce-b1ae-1950a3c013b2 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:28.148203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c00cc3-f735-49ce-b1ae-1950a3c013b2 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:28.154453 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92d204-a18d-470f-bd93-2290a26fd684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:28.155130 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d204-a18d-470f-bd93-2290a26fd684 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:28.155410 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d204-a18d-470f-bd93-2290a26fd684 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:28.155695 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d204-a18d-470f-bd93-2290a26fd684 admin admin] PATH_INFO: `/v3/projects/47dbdfaf1a5c4cc7905fa594f8937e80/users/67d9046e105a4ef68729b0063b78bd57/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:28.174097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92d204-a18d-470f-bd93-2290a26fd684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47dbdfaf1a5c4cc7905fa594f8937e80, user_id=67d9046e105a4ef68729b0063b78bd57, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:28.177221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92d204-a18d-470f-bd93-2290a26fd684 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:28.238903 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1390/2819] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:08:28 2022] PUT /identity/v3/projects/47dbdfaf1a5c4cc7905fa594f8937e80/users/67d9046e105a4ef68729b0063b78bd57/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:08:28.242848 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c491f3c-94f1-41c3-8ce3-f67d6ca05124 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:28.252803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c491f3c-94f1-41c3-8ce3-f67d6ca05124 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:28.308920 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a0ea7-d733-40aa-8c44-6eb2802e9a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:28.309514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a0ea7-d733-40aa-8c44-6eb2802e9a8f admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:28.309793 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a0ea7-d733-40aa-8c44-6eb2802e9a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:28.310064 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a0ea7-d733-40aa-8c44-6eb2802e9a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:28.310740 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a0ea7-d733-40aa-8c44-6eb2802e9a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:28.312384 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a0ea7-d733-40aa-8c44-6eb2802e9a8f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:28.322828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1430/2820] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:08:28.327079 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92d204-a18d-470f-bd93-2290a26fd684 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:28.338250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92d204-a18d-470f-bd93-2290a26fd684 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:28.343061 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9341e02-e741-4088-99f6-51e3f268d03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:28.343701 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9341e02-e741-4088-99f6-51e3f268d03d admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:28.343981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9341e02-e741-4088-99f6-51e3f268d03d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:28.344250 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9341e02-e741-4088-99f6-51e3f268d03d admin admin] PATH_INFO: `/v3/projects/47dbdfaf1a5c4cc7905fa594f8937e80/users/67d9046e105a4ef68729b0063b78bd57/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:28.346116 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9341e02-e741-4088-99f6-51e3f268d03d admin admin] RBAC: Authorizing `identity:create_grant(project_id=47dbdfaf1a5c4cc7905fa594f8937e80, user_id=67d9046e105a4ef68729b0063b78bd57, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:28.348984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9341e02-e741-4088-99f6-51e3f268d03d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:28.359179 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1391/2821] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:08:28 2022] PUT /identity/v3/projects/47dbdfaf1a5c4cc7905fa594f8937e80/users/67d9046e105a4ef68729b0063b78bd57/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:08:28.369251 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92368dbd-eef9-4409-b4ad-3b45bbd4b71a None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:28.369537 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92368dbd-eef9-4409-b4ad-3b45bbd4b71a None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:28.369929 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92368dbd-eef9-4409-b4ad-3b45bbd4b71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:28.398349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-92368dbd-eef9-4409-b4ad-3b45bbd4b71a None None] MFA Rules not processed for user `67d9046e105a4ef68729b0063b78bd57`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:08:28.399645 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-92368dbd-eef9-4409-b4ad-3b45bbd4b71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:28.412038 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:28.473980 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1431/2822] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:08:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:08:28.485133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9341e02-e741-4088-99f6-51e3f268d03d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:28.486847 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e9341e02-e741-4088-99f6-51e3f268d03d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:08:28.543154 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9341e02-e741-4088-99f6-51e3f268d03d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:28.546731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf029b91-b0f9-46e4-8480-47f817bdcd3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:28.547323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf029b91-b0f9-46e4-8480-47f817bdcd3b service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:28.547598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf029b91-b0f9-46e4-8480-47f817bdcd3b service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:28.547863 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf029b91-b0f9-46e4-8480-47f817bdcd3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:28.557111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf029b91-b0f9-46e4-8480-47f817bdcd3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:28.558545 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf029b91-b0f9-46e4-8480-47f817bdcd3b service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:28.568325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1392/2823] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:28.770473 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92368dbd-eef9-4409-b4ad-3b45bbd4b71a None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:08:28.779962 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92368dbd-eef9-4409-b4ad-3b45bbd4b71a None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:08:28.783520 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dffee8-9a97-41d4-adcc-2b083c88ec68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:08:28.787839 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dffee8-9a97-41d4-adcc-2b083c88ec68 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:08:28.787839 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dffee8-9a97-41d4-adcc-2b083c88ec68 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:08:28.787839 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dffee8-9a97-41d4-adcc-2b083c88ec68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:08:28.844059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dffee8-9a97-41d4-adcc-2b083c88ec68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:08:28.845753 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dffee8-9a97-41d4-adcc-2b083c88ec68 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:08:28.855755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1432/2824] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:08:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:08:38.563610 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:08:38.851755 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=50 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:12.672329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf029b91-b0f9-46e4-8480-47f817bdcd3b service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:12.735026 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf029b91-b0f9-46e4-8480-47f817bdcd3b service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:12.739752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:12.740376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:12.740655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:12.740928 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] PATH_INFO: `/v3/users/8b76119bd626453a859c8716605d049c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:12.742211 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b76119bd626453a859c8716605d049c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:12.744271 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:12.931795 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b76119bd626453a859c8716605d049c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:13.009578 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b76119bd626453a859c8716605d049c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:13.010193 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b76119bd626453a859c8716605d049c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:13.011783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b76119bd626453a859c8716605d049c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:13.018989 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b76119bd626453a859c8716605d049c'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:13.036049 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1393/2825] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:09:12 2022] DELETE /identity/v3/users/8b76119bd626453a859c8716605d049c => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:13.040501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0feacfd-571d-49fd-ab47-45c125abef9e None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:13.041125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0feacfd-571d-49fd-ab47-45c125abef9e None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:13.041125 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0feacfd-571d-49fd-ab47-45c125abef9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:13.060897 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b0feacfd-571d-49fd-ab47-45c125abef9e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:13.062076 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b0feacfd-571d-49fd-ab47-45c125abef9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:13.118501 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1433/2826] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:13.129011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:13.130429 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:13.188071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc15bc-f873-4fac-bbcc-c2fa1d4110e6 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:13.191548 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d58b71-5690-4cd4-b22c-50bc396cccf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:13.192139 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d58b71-5690-4cd4-b22c-50bc396cccf3 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:13.192416 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d58b71-5690-4cd4-b22c-50bc396cccf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:13.192681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d58b71-5690-4cd4-b22c-50bc396cccf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:13.201900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d58b71-5690-4cd4-b22c-50bc396cccf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:13.203328 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d58b71-5690-4cd4-b22c-50bc396cccf3 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:13.212836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1394/2827] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:13.608281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0feacfd-571d-49fd-ab47-45c125abef9e None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:13.609798 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b0feacfd-571d-49fd-ab47-45c125abef9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:13.622258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0feacfd-571d-49fd-ab47-45c125abef9e None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:13.626785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:13.627416 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:13.627689 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:13.627960 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] PATH_INFO: `/v3/projects/63dc094692fd489ea9d3548f2b555ed7` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:13.629080 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63dc094692fd489ea9d3548f2b555ed7)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:13.630990 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:13.743861 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63dc094692fd489ea9d3548f2b555ed7', 'request_id': 'req-f2772bcb-07a1-4681-bc56-ff225d6a2891'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:13.750013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63dc094692fd489ea9d3548f2b555ed7', 'request_id': 'req-f2772bcb-07a1-4681-bc56-ff225d6a2891'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:13.834896 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63dc094692fd489ea9d3548f2b555ed7', 'request_id': 'req-f2772bcb-07a1-4681-bc56-ff225d6a2891'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:13.847072 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] The token cache is being invalidate because project 63dc094692fd489ea9d3548f2b555ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:09:13.847357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:13.847986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:13.849471 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1434/2828] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:09:13 2022] DELETE /identity/v3/projects/63dc094692fd489ea9d3548f2b555ed7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:13.853627 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d58b71-5690-4cd4-b22c-50bc396cccf3 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:13.864610 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d58b71-5690-4cd4-b22c-50bc396cccf3 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:13.882944 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:13.921588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27e828-1ffd-4408-b6ad-8a79732b73cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:13.922235 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27e828-1ffd-4408-b6ad-8a79732b73cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:13.922536 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27e828-1ffd-4408-b6ad-8a79732b73cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:13.922812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27e828-1ffd-4408-b6ad-8a79732b73cb admin admin] PATH_INFO: `/v3/users/67d9046e105a4ef68729b0063b78bd57` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:13.924420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27e828-1ffd-4408-b6ad-8a79732b73cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=67d9046e105a4ef68729b0063b78bd57)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:13.925782 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27e828-1ffd-4408-b6ad-8a79732b73cb admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:14.141630 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2a27e828-1ffd-4408-b6ad-8a79732b73cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67d9046e105a4ef68729b0063b78bd57'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:14.223540 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2a27e828-1ffd-4408-b6ad-8a79732b73cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67d9046e105a4ef68729b0063b78bd57'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:14.224147 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2a27e828-1ffd-4408-b6ad-8a79732b73cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67d9046e105a4ef68729b0063b78bd57'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:14.224767 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2a27e828-1ffd-4408-b6ad-8a79732b73cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67d9046e105a4ef68729b0063b78bd57'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:14.232291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-2a27e828-1ffd-4408-b6ad-8a79732b73cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67d9046e105a4ef68729b0063b78bd57'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:14.250429 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1395/2829] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:09:13 2022] DELETE /identity/v3/users/67d9046e105a4ef68729b0063b78bd57 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:14.619070 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:14.621196 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:14.634768 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2772bcb-07a1-4681-bc56-ff225d6a2891 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:14.691421 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b32e8-fddb-46a6-82a5-51f078a81ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:14.691982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b32e8-fddb-46a6-82a5-51f078a81ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:14.692259 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b32e8-fddb-46a6-82a5-51f078a81ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:14.692529 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b32e8-fddb-46a6-82a5-51f078a81ebe admin admin] PATH_INFO: `/v3/projects/47dbdfaf1a5c4cc7905fa594f8937e80` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:14.693668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b32e8-fddb-46a6-82a5-51f078a81ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=47dbdfaf1a5c4cc7905fa594f8937e80)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:14.695622 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b32e8-fddb-46a6-82a5-51f078a81ebe admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:14.767893 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:14.784792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-072b32e8-fddb-46a6-82a5-51f078a81ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47dbdfaf1a5c4cc7905fa594f8937e80', 'request_id': 'req-072b32e8-fddb-46a6-82a5-51f078a81ebe'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:14.792700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-072b32e8-fddb-46a6-82a5-51f078a81ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47dbdfaf1a5c4cc7905fa594f8937e80', 'request_id': 'req-072b32e8-fddb-46a6-82a5-51f078a81ebe'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:14.863215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-072b32e8-fddb-46a6-82a5-51f078a81ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47dbdfaf1a5c4cc7905fa594f8937e80', 'request_id': 'req-072b32e8-fddb-46a6-82a5-51f078a81ebe'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:14.874681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-072b32e8-fddb-46a6-82a5-51f078a81ebe admin admin] The token cache is being invalidate because project 47dbdfaf1a5c4cc7905fa594f8937e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:09:14.874967 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-072b32e8-fddb-46a6-82a5-51f078a81ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:14.875589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-072b32e8-fddb-46a6-82a5-51f078a81ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:14.877101 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1435/2830] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:09:14 2022] DELETE /identity/v3/projects/47dbdfaf1a5c4cc7905fa594f8937e80 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:14.881987 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c24d26-fd7d-4cf1-ac13-8fa3e2c3ba90 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:14.882458 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c24d26-fd7d-4cf1-ac13-8fa3e2c3ba90 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:14.882612 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c24d26-fd7d-4cf1-ac13-8fa3e2c3ba90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:14.902842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-d2c24d26-fd7d-4cf1-ac13-8fa3e2c3ba90 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:14.904047 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-d2c24d26-fd7d-4cf1-ac13-8fa3e2c3ba90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:14.963431 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1396/2831] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:14.971619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42388bc2-8464-4da3-aa2a-e167b2d84dd7 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:14.971903 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42388bc2-8464-4da3-aa2a-e167b2d84dd7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:14.972175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42388bc2-8464-4da3-aa2a-e167b2d84dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:14.992031 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-42388bc2-8464-4da3-aa2a-e167b2d84dd7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:14.993212 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-42388bc2-8464-4da3-aa2a-e167b2d84dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:15.001652 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1436/2832] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:15.005868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c24d26-fd7d-4cf1-ac13-8fa3e2c3ba90 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:15.016789 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c24d26-fd7d-4cf1-ac13-8fa3e2c3ba90 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:15.021334 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e771e0-3752-4342-a551-9ebc27894e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:15.021929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e771e0-3752-4342-a551-9ebc27894e7f admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:15.022211 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e771e0-3752-4342-a551-9ebc27894e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:15.022504 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e771e0-3752-4342-a551-9ebc27894e7f admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:15.023151 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e771e0-3752-4342-a551-9ebc27894e7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:09:15.023450 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e771e0-3752-4342-a551-9ebc27894e7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:15.025024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e771e0-3752-4342-a551-9ebc27894e7f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:15.036904 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1397/2833] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:09:15.041061 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42388bc2-8464-4da3-aa2a-e167b2d84dd7 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:15.050958 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42388bc2-8464-4da3-aa2a-e167b2d84dd7 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:15.055583 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d89b71-b678-4cb2-a902-c9b4e1b36345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:15.056171 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d89b71-b678-4cb2-a902-c9b4e1b36345 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:15.056448 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d89b71-b678-4cb2-a902-c9b4e1b36345 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:15.056718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d89b71-b678-4cb2-a902-c9b4e1b36345 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:15.057387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d89b71-b678-4cb2-a902-c9b4e1b36345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:15.059916 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d89b71-b678-4cb2-a902-c9b4e1b36345 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:15.143247 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1437/2834] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:09:15 2022] POST /identity/v3/projects => generated 409 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:09:15.147463 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e771e0-3752-4342-a551-9ebc27894e7f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:15.158235 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e771e0-3752-4342-a551-9ebc27894e7f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:15.184376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:15.214929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f59adf-ebca-424d-8961-efb51e5a1d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:15.215527 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f59adf-ebca-424d-8961-efb51e5a1d92 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:15.215806 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f59adf-ebca-424d-8961-efb51e5a1d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:15.216076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f59adf-ebca-424d-8961-efb51e5a1d92 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:15.216694 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f59adf-ebca-424d-8961-efb51e5a1d92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:15.218636 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f59adf-ebca-424d-8961-efb51e5a1d92 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:15.281008 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1398/2835] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:09:15 2022] POST /identity/v3/users => generated 305 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:09:15.285206 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d89b71-b678-4cb2-a902-c9b4e1b36345 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:15.296201 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d89b71-b678-4cb2-a902-c9b4e1b36345 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:15.300878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09c839-c03c-49db-a97a-a55d765260b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:15.301469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09c839-c03c-49db-a97a-a55d765260b8 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:15.301751 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09c839-c03c-49db-a97a-a55d765260b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:15.302025 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09c839-c03c-49db-a97a-a55d765260b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:15.302699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09c839-c03c-49db-a97a-a55d765260b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:15.304268 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09c839-c03c-49db-a97a-a55d765260b8 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:15.314078 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1438/2836] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:09:15.317999 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f59adf-ebca-424d-8961-efb51e5a1d92 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:15.328005 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f59adf-ebca-424d-8961-efb51e5a1d92 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:15.332580 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e746294-c5cd-4042-b3ab-7178d3d983db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:15.333165 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e746294-c5cd-4042-b3ab-7178d3d983db admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:15.333442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e746294-c5cd-4042-b3ab-7178d3d983db admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:15.333710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e746294-c5cd-4042-b3ab-7178d3d983db admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:15.334337 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e746294-c5cd-4042-b3ab-7178d3d983db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:15.336291 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e746294-c5cd-4042-b3ab-7178d3d983db admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:15.348175 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1399/2837] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:09:15.352386 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09c839-c03c-49db-a97a-a55d765260b8 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:15.362860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09c839-c03c-49db-a97a-a55d765260b8 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:15.367455 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4be001-be34-4990-a092-210d8debbb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:15.368097 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4be001-be34-4990-a092-210d8debbb89 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:15.368377 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4be001-be34-4990-a092-210d8debbb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:15.368654 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4be001-be34-4990-a092-210d8debbb89 admin admin] PATH_INFO: `/v3/projects/cb9701c656864e96898bc6f1c8c93bc5/users/dfda6f2025ed45259260fa0b7000fd66/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:15.383946 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4be001-be34-4990-a092-210d8debbb89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb9701c656864e96898bc6f1c8c93bc5, user_id=dfda6f2025ed45259260fa0b7000fd66, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:15.386850 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4be001-be34-4990-a092-210d8debbb89 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:15.441074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1439/2838] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:09:15 2022] PUT /identity/v3/projects/cb9701c656864e96898bc6f1c8c93bc5/users/dfda6f2025ed45259260fa0b7000fd66/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:09:15.451652 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f0404-6165-4d2d-81cb-80cbd74525ca None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:15.451939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f0404-6165-4d2d-81cb-80cbd74525ca None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:15.452216 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f0404-6165-4d2d-81cb-80cbd74525ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:15.481497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-de9f0404-6165-4d2d-81cb-80cbd74525ca None None] MFA Rules not processed for user `dfda6f2025ed45259260fa0b7000fd66`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:15.482661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-de9f0404-6165-4d2d-81cb-80cbd74525ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:15.546673 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1400/2839] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:15 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:15.551188 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4be001-be34-4990-a092-210d8debbb89 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:15.611838 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4be001-be34-4990-a092-210d8debbb89 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:15.616440 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edf836-d9d3-47e7-b957-60ea5523d706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:15.617032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edf836-d9d3-47e7-b957-60ea5523d706 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:15.617307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edf836-d9d3-47e7-b957-60ea5523d706 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:15.617574 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edf836-d9d3-47e7-b957-60ea5523d706 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:15.618192 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edf836-d9d3-47e7-b957-60ea5523d706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:15.620109 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edf836-d9d3-47e7-b957-60ea5523d706 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:15.743233 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1440/2840] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:09:15 2022] POST /identity/v3/projects => generated 409 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:09:15.747285 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f0404-6165-4d2d-81cb-80cbd74525ca None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:15.758090 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f0404-6165-4d2d-81cb-80cbd74525ca None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:15.814438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8392d2-ce93-4b32-a186-8503e13dcc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:15.815067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8392d2-ce93-4b32-a186-8503e13dcc3f admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:15.815349 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8392d2-ce93-4b32-a186-8503e13dcc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:15.815642 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8392d2-ce93-4b32-a186-8503e13dcc3f admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:15.816274 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8392d2-ce93-4b32-a186-8503e13dcc3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:15.818274 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8392d2-ce93-4b32-a186-8503e13dcc3f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:15.903043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1401/2841] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:09:15 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:09:15.907044 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edf836-d9d3-47e7-b957-60ea5523d706 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:15.917396 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edf836-d9d3-47e7-b957-60ea5523d706 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:15.922159 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bc62c-56bb-492f-8573-4200e10952cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:15.922785 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bc62c-56bb-492f-8573-4200e10952cd admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:15.923085 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bc62c-56bb-492f-8573-4200e10952cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:15.923335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bc62c-56bb-492f-8573-4200e10952cd admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:15.923966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bc62c-56bb-492f-8573-4200e10952cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:15.925524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bc62c-56bb-492f-8573-4200e10952cd admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:15.936203 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1441/2842] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:09:15.940234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8392d2-ce93-4b32-a186-8503e13dcc3f admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:15.950852 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8392d2-ce93-4b32-a186-8503e13dcc3f admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:15.955421 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb029201-4c1a-46e9-a2a6-bd5f770429a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:15.958055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb029201-4c1a-46e9-a2a6-bd5f770429a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:15.958055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb029201-4c1a-46e9-a2a6-bd5f770429a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:15.958055 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb029201-4c1a-46e9-a2a6-bd5f770429a0 admin admin] PATH_INFO: `/v3/projects/aa7871a6448b4942a61ad010b8268440/users/20af5e65b9084b1eaa96f41a354754e8/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:15.971438 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb029201-4c1a-46e9-a2a6-bd5f770429a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa7871a6448b4942a61ad010b8268440, user_id=20af5e65b9084b1eaa96f41a354754e8, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:15.974060 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb029201-4c1a-46e9-a2a6-bd5f770429a0 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:16.051547 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1402/2843] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:09:15 2022] PUT /identity/v3/projects/aa7871a6448b4942a61ad010b8268440/users/20af5e65b9084b1eaa96f41a354754e8/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:16.055403 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bc62c-56bb-492f-8573-4200e10952cd admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:16.065982 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bc62c-56bb-492f-8573-4200e10952cd admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:16.086457 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:16.124099 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff9cfe-a62b-4a6f-bded-de6ece00ef80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:16.124781 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff9cfe-a62b-4a6f-bded-de6ece00ef80 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:16.125000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff9cfe-a62b-4a6f-bded-de6ece00ef80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:16.125271 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff9cfe-a62b-4a6f-bded-de6ece00ef80 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:16.125911 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff9cfe-a62b-4a6f-bded-de6ece00ef80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:16.127518 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff9cfe-a62b-4a6f-bded-de6ece00ef80 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:16.138335 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1442/2844] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 13:09:16.142185 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb029201-4c1a-46e9-a2a6-bd5f770429a0 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:16.147657 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:16.153021 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb029201-4c1a-46e9-a2a6-bd5f770429a0 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:16.157680 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e75f3-648c-41c8-9b45-26cfc3367fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:16.158323 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e75f3-648c-41c8-9b45-26cfc3367fdc admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:16.158648 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e75f3-648c-41c8-9b45-26cfc3367fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:16.158926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e75f3-648c-41c8-9b45-26cfc3367fdc admin admin] PATH_INFO: `/v3/projects/aa7871a6448b4942a61ad010b8268440/users/20af5e65b9084b1eaa96f41a354754e8/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:16.160941 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e75f3-648c-41c8-9b45-26cfc3367fdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa7871a6448b4942a61ad010b8268440, user_id=20af5e65b9084b1eaa96f41a354754e8, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:16.163763 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e75f3-648c-41c8-9b45-26cfc3367fdc admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:16.177131 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1403/2845] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:09:16 2022] PUT /identity/v3/projects/aa7871a6448b4942a61ad010b8268440/users/20af5e65b9084b1eaa96f41a354754e8/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:16.189189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef54d9-2fda-4cb1-b379-75c5cf4fa466 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:16.189475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef54d9-2fda-4cb1-b379-75c5cf4fa466 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:16.189749 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef54d9-2fda-4cb1-b379-75c5cf4fa466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:16.218312 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-feef54d9-2fda-4cb1-b379-75c5cf4fa466 None None] MFA Rules not processed for user `20af5e65b9084b1eaa96f41a354754e8`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:16.219598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-feef54d9-2fda-4cb1-b379-75c5cf4fa466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:16.285540 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1443/2846] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:16 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:16.296688 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e75f3-648c-41c8-9b45-26cfc3367fdc admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:16.298425 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-e00e75f3-648c-41c8-9b45-26cfc3367fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:16.355189 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e75f3-648c-41c8-9b45-26cfc3367fdc admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:16.359369 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059c0f8-2050-4db4-ac3e-c36cee502b10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:16.359971 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059c0f8-2050-4db4-ac3e-c36cee502b10 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:16.360335 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059c0f8-2050-4db4-ac3e-c36cee502b10 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:16.360528 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059c0f8-2050-4db4-ac3e-c36cee502b10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:16.369459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059c0f8-2050-4db4-ac3e-c36cee502b10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:16.370922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059c0f8-2050-4db4-ac3e-c36cee502b10 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:16.381163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1404/2847] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:16 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:16.752726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef54d9-2fda-4cb1-b379-75c5cf4fa466 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:16.762617 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef54d9-2fda-4cb1-b379-75c5cf4fa466 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:16.766860 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5db64-fbdf-48c0-94d9-26236ddc42db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:16.767487 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5db64-fbdf-48c0-94d9-26236ddc42db service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:16.767766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5db64-fbdf-48c0-94d9-26236ddc42db service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:16.768032 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5db64-fbdf-48c0-94d9-26236ddc42db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:16.823019 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5db64-fbdf-48c0-94d9-26236ddc42db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:16.824430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5db64-fbdf-48c0-94d9-26236ddc42db service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:16.833446 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1444/2848] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:16 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:25.651449 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059c0f8-2050-4db4-ac3e-c36cee502b10 service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:25.713017 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059c0f8-2050-4db4-ac3e-c36cee502b10 service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:25.718011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:25.718651 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:25.718955 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:25.719231 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] PATH_INFO: `/v3/users/dfda6f2025ed45259260fa0b7000fd66` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:25.720336 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfda6f2025ed45259260fa0b7000fd66)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:25.722233 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:25.893303 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfda6f2025ed45259260fa0b7000fd66'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:25.953924 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfda6f2025ed45259260fa0b7000fd66'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:25.954552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfda6f2025ed45259260fa0b7000fd66'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:25.955152 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfda6f2025ed45259260fa0b7000fd66'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:25.962516 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfda6f2025ed45259260fa0b7000fd66'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:25.980112 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1405/2849] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:09:25 2022] DELETE /identity/v3/users/dfda6f2025ed45259260fa0b7000fd66 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:25.985277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026622e-c34b-40d4-b45f-35c2f7860130 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:25.985786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026622e-c34b-40d4-b45f-35c2f7860130 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:25.985905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026622e-c34b-40d4-b45f-35c2f7860130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:26.005102 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d026622e-c34b-40d4-b45f-35c2f7860130 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:26.006288 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d026622e-c34b-40d4-b45f-35c2f7860130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:26.046255 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=100 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:26.064474 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1445/2850] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:26.074248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:26.076171 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:26.089663 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:26.134209 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69506c48-7b0c-41b7-b91b-ef2ba5b81c96 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:26.138297 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00decd4a-a434-4f95-9d6f-805557ffbf28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:26.138995 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00decd4a-a434-4f95-9d6f-805557ffbf28 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:26.145805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00decd4a-a434-4f95-9d6f-805557ffbf28 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:26.145805 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00decd4a-a434-4f95-9d6f-805557ffbf28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:26.148259 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00decd4a-a434-4f95-9d6f-805557ffbf28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:26.149659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00decd4a-a434-4f95-9d6f-805557ffbf28 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:26.159555 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1406/2851] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:26.556395 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026622e-c34b-40d4-b45f-35c2f7860130 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:26.557822 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d026622e-c34b-40d4-b45f-35c2f7860130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:26.570122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026622e-c34b-40d4-b45f-35c2f7860130 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:26.574748 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:26.576049 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:26.576272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:26.576547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] PATH_INFO: `/v3/projects/cb9701c656864e96898bc6f1c8c93bc5` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:26.577686 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb9701c656864e96898bc6f1c8c93bc5)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:26.579614 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:26.680173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb9701c656864e96898bc6f1c8c93bc5', 'request_id': 'req-0baaa73d-b9fd-4f21-a414-270ade52e329'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:26.687026 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb9701c656864e96898bc6f1c8c93bc5', 'request_id': 'req-0baaa73d-b9fd-4f21-a414-270ade52e329'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:26.765551 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb9701c656864e96898bc6f1c8c93bc5', 'request_id': 'req-0baaa73d-b9fd-4f21-a414-270ade52e329'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:26.776146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] The token cache is being invalidate because project cb9701c656864e96898bc6f1c8c93bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:09:26.781015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:26.781015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:26.781015 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1446/2852] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:09:26 2022] DELETE /identity/v3/projects/cb9701c656864e96898bc6f1c8c93bc5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:26.784435 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00decd4a-a434-4f95-9d6f-805557ffbf28 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:26.795176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00decd4a-a434-4f95-9d6f-805557ffbf28 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:26.851939 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c662ed-015c-477b-b594-3f9a79aab5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:26.852489 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c662ed-015c-477b-b594-3f9a79aab5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:26.852768 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c662ed-015c-477b-b594-3f9a79aab5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:26.853040 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c662ed-015c-477b-b594-3f9a79aab5e6 admin admin] PATH_INFO: `/v3/users/20af5e65b9084b1eaa96f41a354754e8` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:26.854140 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c662ed-015c-477b-b594-3f9a79aab5e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20af5e65b9084b1eaa96f41a354754e8)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:26.856071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c662ed-015c-477b-b594-3f9a79aab5e6 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:27.058865 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a9c662ed-015c-477b-b594-3f9a79aab5e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20af5e65b9084b1eaa96f41a354754e8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:27.125895 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a9c662ed-015c-477b-b594-3f9a79aab5e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20af5e65b9084b1eaa96f41a354754e8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:27.126557 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a9c662ed-015c-477b-b594-3f9a79aab5e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20af5e65b9084b1eaa96f41a354754e8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:27.127165 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a9c662ed-015c-477b-b594-3f9a79aab5e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20af5e65b9084b1eaa96f41a354754e8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:27.134659 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-a9c662ed-015c-477b-b594-3f9a79aab5e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20af5e65b9084b1eaa96f41a354754e8'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:27.152868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1407/2853] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:09:26 2022] DELETE /identity/v3/users/20af5e65b9084b1eaa96f41a354754e8 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:27.555449 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:27.556863 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:27.577174 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaa73d-b9fd-4f21-a414-270ade52e329 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:27.633460 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:27.634094 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:27.634405 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:27.634707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0 admin admin] PATH_INFO: `/v3/projects/aa7871a6448b4942a61ad010b8268440` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:27.635844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa7871a6448b4942a61ad010b8268440)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:27.637757 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:27.766571 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa7871a6448b4942a61ad010b8268440', 'request_id': 'req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:27.772603 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa7871a6448b4942a61ad010b8268440', 'request_id': 'req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:27.855664 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa7871a6448b4942a61ad010b8268440', 'request_id': 'req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:27.863514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:27.868211 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0 admin admin] The token cache is being invalidate because project aa7871a6448b4942a61ad010b8268440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:09:27.868499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:27.869130 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-0e6c9551-ebdd-43ce-a21b-9c75f5f9f4e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:27.870736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1447/2854] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:09:27 2022] DELETE /identity/v3/projects/aa7871a6448b4942a61ad010b8268440 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:27.875506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83835de-3b2c-48b1-a4a7-37bede56cac8 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:27.875981 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83835de-3b2c-48b1-a4a7-37bede56cac8 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:27.876106 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83835de-3b2c-48b1-a4a7-37bede56cac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:27.896693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-c83835de-3b2c-48b1-a4a7-37bede56cac8 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:27.897874 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c83835de-3b2c-48b1-a4a7-37bede56cac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:27.925994 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:27.955111 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1408/2855] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:27.963199 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0c7a7-ada6-4dd6-83b0-3a94497e9250 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:27.963487 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0c7a7-ada6-4dd6-83b0-3a94497e9250 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:27.963761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0c7a7-ada6-4dd6-83b0-3a94497e9250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:27.985128 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-aab0c7a7-ada6-4dd6-83b0-3a94497e9250 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:27.986314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-aab0c7a7-ada6-4dd6-83b0-3a94497e9250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:27.994172 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1448/2856] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:27.998647 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83835de-3b2c-48b1-a4a7-37bede56cac8 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:28.008178 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83835de-3b2c-48b1-a4a7-37bede56cac8 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:28.012917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18b541-0b7a-42b0-8091-e53a0c4c4ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:28.014614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18b541-0b7a-42b0-8091-e53a0c4c4ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:28.014614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18b541-0b7a-42b0-8091-e53a0c4c4ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:28.014614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18b541-0b7a-42b0-8091-e53a0c4c4ce2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:28.015176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18b541-0b7a-42b0-8091-e53a0c4c4ce2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:09:28.015478 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18b541-0b7a-42b0-8091-e53a0c4c4ce2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:28.017095 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18b541-0b7a-42b0-8091-e53a0c4c4ce2 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:28.028267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1409/2857] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:09:28.032991 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0c7a7-ada6-4dd6-83b0-3a94497e9250 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:28.041913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0c7a7-ada6-4dd6-83b0-3a94497e9250 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:28.047253 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebc11e-166f-48c2-986e-290faf7980e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:28.047840 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebc11e-166f-48c2-986e-290faf7980e3 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:28.048122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebc11e-166f-48c2-986e-290faf7980e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:28.048391 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebc11e-166f-48c2-986e-290faf7980e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:28.049010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebc11e-166f-48c2-986e-290faf7980e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:28.050921 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebc11e-166f-48c2-986e-290faf7980e3 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:28.131002 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1449/2858] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:09:28 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:09:28.135309 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18b541-0b7a-42b0-8091-e53a0c4c4ce2 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:28.145286 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18b541-0b7a-42b0-8091-e53a0c4c4ce2 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:28.207300 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113ad22-a427-4b3f-adc7-a6bdd7878ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:28.214792 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113ad22-a427-4b3f-adc7-a6bdd7878ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:28.214792 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113ad22-a427-4b3f-adc7-a6bdd7878ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:28.214792 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113ad22-a427-4b3f-adc7-a6bdd7878ce3 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:28.214792 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113ad22-a427-4b3f-adc7-a6bdd7878ce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:28.214792 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113ad22-a427-4b3f-adc7-a6bdd7878ce3 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:28.296420 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1410/2859] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:09:28 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:09:28.300441 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebc11e-166f-48c2-986e-290faf7980e3 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:28.311107 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebc11e-166f-48c2-986e-290faf7980e3 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:28.315772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f875e-9755-4103-b21a-d8903b1d2ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:28.316363 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f875e-9755-4103-b21a-d8903b1d2ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:28.316642 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f875e-9755-4103-b21a-d8903b1d2ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:28.316915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f875e-9755-4103-b21a-d8903b1d2ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:28.317594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f875e-9755-4103-b21a-d8903b1d2ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:28.319214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f875e-9755-4103-b21a-d8903b1d2ee1 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:28.330153 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1450/2860] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:09:28.333686 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113ad22-a427-4b3f-adc7-a6bdd7878ce3 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:28.344871 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113ad22-a427-4b3f-adc7-a6bdd7878ce3 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:28.349442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e37b4-48b4-4cef-8a5a-5df53ed0925e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:28.350024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e37b4-48b4-4cef-8a5a-5df53ed0925e admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:28.350302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e37b4-48b4-4cef-8a5a-5df53ed0925e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:28.350699 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e37b4-48b4-4cef-8a5a-5df53ed0925e admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:28.351327 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e37b4-48b4-4cef-8a5a-5df53ed0925e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:28.352897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e37b4-48b4-4cef-8a5a-5df53ed0925e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:28.363558 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1411/2861] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:09:28.367699 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f875e-9755-4103-b21a-d8903b1d2ee1 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:28.377995 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f875e-9755-4103-b21a-d8903b1d2ee1 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:28.382584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaca2b5-1d2a-4fbb-b4e2-b50e59b9a440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:28.383222 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaca2b5-1d2a-4fbb-b4e2-b50e59b9a440 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:28.383500 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaca2b5-1d2a-4fbb-b4e2-b50e59b9a440 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:28.383769 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaca2b5-1d2a-4fbb-b4e2-b50e59b9a440 admin admin] PATH_INFO: `/v3/projects/b062bbb9dba14efd9ae21eafde025d2a/users/6e6e92b1f3804fefb5365cf14722b454/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:28.398750 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaca2b5-1d2a-4fbb-b4e2-b50e59b9a440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b062bbb9dba14efd9ae21eafde025d2a, user_id=6e6e92b1f3804fefb5365cf14722b454, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:28.401357 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaca2b5-1d2a-4fbb-b4e2-b50e59b9a440 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:28.452180 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1451/2862] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:09:28 2022] PUT /identity/v3/projects/b062bbb9dba14efd9ae21eafde025d2a/users/6e6e92b1f3804fefb5365cf14722b454/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:28.524067 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b93063-e99f-49f7-9d2e-cc482134603d None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:28.524490 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b93063-e99f-49f7-9d2e-cc482134603d None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:28.524790 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b93063-e99f-49f7-9d2e-cc482134603d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:28.554832 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-f5b93063-e99f-49f7-9d2e-cc482134603d None None] MFA Rules not processed for user `6e6e92b1f3804fefb5365cf14722b454`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:28.556008 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f5b93063-e99f-49f7-9d2e-cc482134603d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:28.619827 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1412/2863] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:28.630830 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaca2b5-1d2a-4fbb-b4e2-b50e59b9a440 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:28.632566 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-eaaca2b5-1d2a-4fbb-b4e2-b50e59b9a440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:28.688794 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaca2b5-1d2a-4fbb-b4e2-b50e59b9a440 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:28.692410 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89001f7-e80a-4d14-a88a-3b8ab29847b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:28.693003 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89001f7-e80a-4d14-a88a-3b8ab29847b8 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:28.693281 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89001f7-e80a-4d14-a88a-3b8ab29847b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:28.693553 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89001f7-e80a-4d14-a88a-3b8ab29847b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:28.701712 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89001f7-e80a-4d14-a88a-3b8ab29847b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:28.703156 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89001f7-e80a-4d14-a88a-3b8ab29847b8 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:28.711983 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1452/2864] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:32.738117 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b93063-e99f-49f7-9d2e-cc482134603d None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:32.769009 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:32.800492 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b93063-e99f-49f7-9d2e-cc482134603d None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:32.804980 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:32.805602 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:32.805881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:32.806153 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] PATH_INFO: `/v3/users/6e6e92b1f3804fefb5365cf14722b454` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:32.807294 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6e92b1f3804fefb5365cf14722b454)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:32.809224 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:33.034638 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6e92b1f3804fefb5365cf14722b454'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:33.119868 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6e92b1f3804fefb5365cf14722b454'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:33.120539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6e92b1f3804fefb5365cf14722b454'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:33.121150 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6e92b1f3804fefb5365cf14722b454'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:33.130282 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6e92b1f3804fefb5365cf14722b454'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:33.150811 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1413/2865] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:09:32 2022] DELETE /identity/v3/users/6e6e92b1f3804fefb5365cf14722b454 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:33.157314 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf68c4b-3700-4586-b808-26ca92aa2f87 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:33.157809 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf68c4b-3700-4586-b808-26ca92aa2f87 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:33.157925 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf68c4b-3700-4586-b808-26ca92aa2f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:33.176803 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-dbf68c4b-3700-4586-b808-26ca92aa2f87 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:33.177998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-dbf68c4b-3700-4586-b808-26ca92aa2f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:33.236606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1453/2866] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:33.246236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:33.248270 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:33.304248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2c2e9-ff55-49fc-95e5-03518b8d4f46 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:33.307706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c72938-fdca-440d-a934-009236e31c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:33.308301 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c72938-fdca-440d-a934-009236e31c1a service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:33.308579 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c72938-fdca-440d-a934-009236e31c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:33.308853 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c72938-fdca-440d-a934-009236e31c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:33.318226 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c72938-fdca-440d-a934-009236e31c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:33.319733 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c72938-fdca-440d-a934-009236e31c1a service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:33.329671 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1414/2867] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:33.710030 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf68c4b-3700-4586-b808-26ca92aa2f87 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:33.711761 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-dbf68c4b-3700-4586-b808-26ca92aa2f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:33.725619 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf68c4b-3700-4586-b808-26ca92aa2f87 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:33.729177 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9e5f1-1542-4c40-81ba-19262014fdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:33.729819 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9e5f1-1542-4c40-81ba-19262014fdf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:33.730097 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9e5f1-1542-4c40-81ba-19262014fdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:33.730368 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9e5f1-1542-4c40-81ba-19262014fdf1 admin admin] PATH_INFO: `/v3/projects/b062bbb9dba14efd9ae21eafde025d2a` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:33.732126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9e5f1-1542-4c40-81ba-19262014fdf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b062bbb9dba14efd9ae21eafde025d2a)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:33.734558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9e5f1-1542-4c40-81ba-19262014fdf1 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:33.812337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:33.823666 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-79f9e5f1-1542-4c40-81ba-19262014fdf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b062bbb9dba14efd9ae21eafde025d2a', 'request_id': 'req-79f9e5f1-1542-4c40-81ba-19262014fdf1'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:33.829675 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-79f9e5f1-1542-4c40-81ba-19262014fdf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b062bbb9dba14efd9ae21eafde025d2a', 'request_id': 'req-79f9e5f1-1542-4c40-81ba-19262014fdf1'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:33.902667 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-79f9e5f1-1542-4c40-81ba-19262014fdf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b062bbb9dba14efd9ae21eafde025d2a', 'request_id': 'req-79f9e5f1-1542-4c40-81ba-19262014fdf1'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:33.913915 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-79f9e5f1-1542-4c40-81ba-19262014fdf1 admin admin] The token cache is being invalidate because project b062bbb9dba14efd9ae21eafde025d2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88041) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:09:33.914207 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-79f9e5f1-1542-4c40-81ba-19262014fdf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:33.914861 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-79f9e5f1-1542-4c40-81ba-19262014fdf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:33.916366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1454/2868] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:09:33 2022] DELETE /identity/v3/projects/b062bbb9dba14efd9ae21eafde025d2a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:33.921378 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc69789-f696-4f0e-a429-ca5f0e82bbac None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:33.921861 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc69789-f696-4f0e-a429-ca5f0e82bbac None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:33.921991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc69789-f696-4f0e-a429-ca5f0e82bbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:33.941681 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-1fc69789-f696-4f0e-a429-ca5f0e82bbac None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:33.942912 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-1fc69789-f696-4f0e-a429-ca5f0e82bbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:33.987107 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:34.001646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1415/2869] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:34.009063 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fc9d7-4736-42e9-a845-db31574088ef None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:34.009346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fc9d7-4736-42e9-a845-db31574088ef None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:34.010685 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fc9d7-4736-42e9-a845-db31574088ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:34.029592 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-c62fc9d7-4736-42e9-a845-db31574088ef None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:34.033728 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-c62fc9d7-4736-42e9-a845-db31574088ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:34.038597 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1455/2870] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:34.042835 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc69789-f696-4f0e-a429-ca5f0e82bbac None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:34.052758 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc69789-f696-4f0e-a429-ca5f0e82bbac None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:34.057327 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d75ff-3313-467f-8e4f-8e133bd4a4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:34.057922 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d75ff-3313-467f-8e4f-8e133bd4a4ab admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:34.058203 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d75ff-3313-467f-8e4f-8e133bd4a4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:34.058477 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d75ff-3313-467f-8e4f-8e133bd4a4ab admin admin] PATH_INFO: `/v3/domains` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:34.059152 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d75ff-3313-467f-8e4f-8e133bd4a4ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88042) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:09:34.060042 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d75ff-3313-467f-8e4f-8e133bd4a4ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:34.062016 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d75ff-3313-467f-8e4f-8e133bd4a4ab admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:34.079609 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1416/2871] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:09:34.082558 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fc9d7-4736-42e9-a845-db31574088ef None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:34.092653 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fc9d7-4736-42e9-a845-db31574088ef None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:34.097308 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656aa72-0d6d-4507-b48c-85ca49563c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:34.102668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656aa72-0d6d-4507-b48c-85ca49563c56 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:34.102668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656aa72-0d6d-4507-b48c-85ca49563c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:34.102668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656aa72-0d6d-4507-b48c-85ca49563c56 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:34.102668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656aa72-0d6d-4507-b48c-85ca49563c56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:34.102668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656aa72-0d6d-4507-b48c-85ca49563c56 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:34.182828 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1456/2872] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:09:34 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:09:34.187473 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d75ff-3313-467f-8e4f-8e133bd4a4ab admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:34.197422 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d75ff-3313-467f-8e4f-8e133bd4a4ab admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:34.254329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e3f63-108a-4337-9dd9-45036fbcf11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:34.254963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e3f63-108a-4337-9dd9-45036fbcf11e admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:34.255238 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e3f63-108a-4337-9dd9-45036fbcf11e admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:34.255506 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e3f63-108a-4337-9dd9-45036fbcf11e admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:34.256128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e3f63-108a-4337-9dd9-45036fbcf11e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:34.258011 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e3f63-108a-4337-9dd9-45036fbcf11e admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:34.342611 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1417/2873] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:09:34 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:09:34.346575 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656aa72-0d6d-4507-b48c-85ca49563c56 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:34.357356 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656aa72-0d6d-4507-b48c-85ca49563c56 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:34.362053 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730db5b-ff63-4b19-874d-151e8ac0e158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:34.362669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730db5b-ff63-4b19-874d-151e8ac0e158 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:34.362949 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730db5b-ff63-4b19-874d-151e8ac0e158 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:34.363227 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730db5b-ff63-4b19-874d-151e8ac0e158 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:34.363894 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730db5b-ff63-4b19-874d-151e8ac0e158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:34.365467 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730db5b-ff63-4b19-874d-151e8ac0e158 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:34.376875 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1457/2874] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:09:34.380843 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e3f63-108a-4337-9dd9-45036fbcf11e admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:34.391234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e3f63-108a-4337-9dd9-45036fbcf11e admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:34.395857 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42737bc4-3472-485d-86e1-40dcf1d7b7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:34.396446 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42737bc4-3472-485d-86e1-40dcf1d7b7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:34.396728 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42737bc4-3472-485d-86e1-40dcf1d7b7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:34.397003 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42737bc4-3472-485d-86e1-40dcf1d7b7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:34.397646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42737bc4-3472-485d-86e1-40dcf1d7b7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:34.399266 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42737bc4-3472-485d-86e1-40dcf1d7b7e8 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:34.411483 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1418/2875] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:09:34.415463 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730db5b-ff63-4b19-874d-151e8ac0e158 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:34.426311 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730db5b-ff63-4b19-874d-151e8ac0e158 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:34.431066 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadf96d-3937-460f-9e31-0abeeac80457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:34.431707 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadf96d-3937-460f-9e31-0abeeac80457 admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:34.431985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadf96d-3937-460f-9e31-0abeeac80457 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:34.432258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadf96d-3937-460f-9e31-0abeeac80457 admin admin] PATH_INFO: `/v3/projects/7de0763651344f19911cfd82baa23add/users/fcdc0ed4c88543dfb12c41c130aac500/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:34.447231 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadf96d-3937-460f-9e31-0abeeac80457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7de0763651344f19911cfd82baa23add, user_id=fcdc0ed4c88543dfb12c41c130aac500, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:34.449890 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadf96d-3937-460f-9e31-0abeeac80457 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:34.536879 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1458/2876] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:09:34 2022] PUT /identity/v3/projects/7de0763651344f19911cfd82baa23add/users/fcdc0ed4c88543dfb12c41c130aac500/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:34.547377 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c37a09-81c1-42a6-9136-fa08b3a2f6a9 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:34.547658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c37a09-81c1-42a6-9136-fa08b3a2f6a9 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:34.547929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c37a09-81c1-42a6-9136-fa08b3a2f6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:34.575367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-c7c37a09-81c1-42a6-9136-fa08b3a2f6a9 None None] MFA Rules not processed for user `fcdc0ed4c88543dfb12c41c130aac500`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:34.576582 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c7c37a09-81c1-42a6-9136-fa08b3a2f6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:34.639267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1419/2877] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:34.643851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadf96d-3937-460f-9e31-0abeeac80457 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:34.703697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadf96d-3937-460f-9e31-0abeeac80457 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:34.708265 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee082a0-8643-4f66-83a3-42b87eed5b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:34.708857 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee082a0-8643-4f66-83a3-42b87eed5b5c admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:34.709132 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee082a0-8643-4f66-83a3-42b87eed5b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:34.709398 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee082a0-8643-4f66-83a3-42b87eed5b5c admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:34.710022 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee082a0-8643-4f66-83a3-42b87eed5b5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:34.712173 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee082a0-8643-4f66-83a3-42b87eed5b5c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:34.793906 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1459/2878] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:09:34 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:09:34.799883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c37a09-81c1-42a6-9136-fa08b3a2f6a9 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:34.809576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c37a09-81c1-42a6-9136-fa08b3a2f6a9 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:34.853097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:34.866318 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf7879-d443-49e9-8f52-5eafa54f67f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:34.866949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf7879-d443-49e9-8f52-5eafa54f67f6 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:34.867227 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf7879-d443-49e9-8f52-5eafa54f67f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:34.867497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf7879-d443-49e9-8f52-5eafa54f67f6 admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:34.868108 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf7879-d443-49e9-8f52-5eafa54f67f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:34.869984 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf7879-d443-49e9-8f52-5eafa54f67f6 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:34.953972 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1420/2879] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:09:34 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:09:34.958089 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee082a0-8643-4f66-83a3-42b87eed5b5c admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:34.969454 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee082a0-8643-4f66-83a3-42b87eed5b5c admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:34.973997 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b912a-0579-4e2c-a5c8-e7b1f905781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:34.974684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b912a-0579-4e2c-a5c8-e7b1f905781c admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:34.974990 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b912a-0579-4e2c-a5c8-e7b1f905781c admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:34.980631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b912a-0579-4e2c-a5c8-e7b1f905781c admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:34.980631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b912a-0579-4e2c-a5c8-e7b1f905781c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:34.980631 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b912a-0579-4e2c-a5c8-e7b1f905781c admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:34.988697 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1460/2880] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:09:34.992526 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf7879-d443-49e9-8f52-5eafa54f67f6 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:35.002426 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf7879-d443-49e9-8f52-5eafa54f67f6 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:35.007076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a3461-2017-4b6c-81b3-11bed013193f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:35.007605 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3461-2017-4b6c-81b3-11bed013193f admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:35.007888 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3461-2017-4b6c-81b3-11bed013193f admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:35.008160 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3461-2017-4b6c-81b3-11bed013193f admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:35.008810 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a3461-2017-4b6c-81b3-11bed013193f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:35.010443 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a3461-2017-4b6c-81b3-11bed013193f admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:35.020721 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1421/2881] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:09:35.024709 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b912a-0579-4e2c-a5c8-e7b1f905781c admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:35.035517 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b912a-0579-4e2c-a5c8-e7b1f905781c admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:35.040088 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b371eaa-933e-4637-834c-285b40d733dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:35.040722 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b371eaa-933e-4637-834c-285b40d733dc admin admin] REQUEST_METHOD: `PUT` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:35.041000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b371eaa-933e-4637-834c-285b40d733dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:35.041274 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b371eaa-933e-4637-834c-285b40d733dc admin admin] PATH_INFO: `/v3/projects/5d270690524b43269e0120cb3561abf0/users/bc25f538b4384ec583c4c9c71283af59/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:35.056681 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b371eaa-933e-4637-834c-285b40d733dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d270690524b43269e0120cb3561abf0, user_id=bc25f538b4384ec583c4c9c71283af59, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:35.059329 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b371eaa-933e-4637-834c-285b40d733dc admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:35.131889 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1461/2882] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:09:35 2022] PUT /identity/v3/projects/5d270690524b43269e0120cb3561abf0/users/bc25f538b4384ec583c4c9c71283af59/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:35.143759 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ccf4e-d21b-4056-be6b-80f7544da704 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:35.144054 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ccf4e-d21b-4056-be6b-80f7544da704 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:35.144331 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ccf4e-d21b-4056-be6b-80f7544da704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:35.172976 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-406ccf4e-d21b-4056-be6b-80f7544da704 None None] MFA Rules not processed for user `bc25f538b4384ec583c4c9c71283af59`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:35.174164 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-406ccf4e-d21b-4056-be6b-80f7544da704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:35.237848 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1422/2883] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:35.242361 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b371eaa-933e-4637-834c-285b40d733dc admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:35.251241 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:35.303366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b371eaa-933e-4637-834c-285b40d733dc admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:35.308175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045a19f-7be1-417b-b2ba-34d3b70f7cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:35.308786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045a19f-7be1-417b-b2ba-34d3b70f7cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:35.309303 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045a19f-7be1-417b-b2ba-34d3b70f7cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:35.309981 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045a19f-7be1-417b-b2ba-34d3b70f7cb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:35.310726 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045a19f-7be1-417b-b2ba-34d3b70f7cb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:35.312818 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045a19f-7be1-417b-b2ba-34d3b70f7cb7 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:35.412990 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1462/2884] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:09:35 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:09:35.417083 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ccf4e-d21b-4056-be6b-80f7544da704 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:35.428114 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ccf4e-d21b-4056-be6b-80f7544da704 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:35.488074 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dffa19-4e08-4bf4-96b0-18a3372ea12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:35.488598 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dffa19-4e08-4bf4-96b0-18a3372ea12a admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:35.488883 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dffa19-4e08-4bf4-96b0-18a3372ea12a admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:35.489159 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dffa19-4e08-4bf4-96b0-18a3372ea12a admin admin] PATH_INFO: `/v3/users` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:35.489823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dffa19-4e08-4bf4-96b0-18a3372ea12a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:35.491749 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dffa19-4e08-4bf4-96b0-18a3372ea12a admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:35.566119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1423/2885] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:09:35 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:09:35.572747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045a19f-7be1-417b-b2ba-34d3b70f7cb7 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:35.581786 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045a19f-7be1-417b-b2ba-34d3b70f7cb7 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:35.587966 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7219f-5a4a-46a5-b8fd-baa87ac2957f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:35.588589 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7219f-5a4a-46a5-b8fd-baa87ac2957f admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:35.592611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7219f-5a4a-46a5-b8fd-baa87ac2957f admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:35.592611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7219f-5a4a-46a5-b8fd-baa87ac2957f admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:35.592611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7219f-5a4a-46a5-b8fd-baa87ac2957f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:35.592611 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7219f-5a4a-46a5-b8fd-baa87ac2957f admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:35.603189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1463/2886] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:09:35.607414 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dffa19-4e08-4bf4-96b0-18a3372ea12a admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:35.617540 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dffa19-4e08-4bf4-96b0-18a3372ea12a admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:35.622710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbf7ad-1905-4b17-a871-63cd1c85c244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:35.623356 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbf7ad-1905-4b17-a871-63cd1c85c244 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:35.623635 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbf7ad-1905-4b17-a871-63cd1c85c244 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:35.623907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbf7ad-1905-4b17-a871-63cd1c85c244 admin admin] PATH_INFO: `/v3/projects/b23d26bf57cc4c089506d5100840712e/users/598197f58e1d4505a3d1769652e107fc/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:35.638801 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbf7ad-1905-4b17-a871-63cd1c85c244 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23d26bf57cc4c089506d5100840712e, user_id=598197f58e1d4505a3d1769652e107fc, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:35.641452 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbf7ad-1905-4b17-a871-63cd1c85c244 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:35.739121 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1424/2887] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:09:35 2022] PUT /identity/v3/projects/b23d26bf57cc4c089506d5100840712e/users/598197f58e1d4505a3d1769652e107fc/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:09:35.743082 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7219f-5a4a-46a5-b8fd-baa87ac2957f admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:35.755968 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7219f-5a4a-46a5-b8fd-baa87ac2957f admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:35.814672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0407a92-612f-4e77-ba58-f16c8079169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:35.815270 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0407a92-612f-4e77-ba58-f16c8079169b admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:35.815549 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0407a92-612f-4e77-ba58-f16c8079169b admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:35.815823 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0407a92-612f-4e77-ba58-f16c8079169b admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:35.816453 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0407a92-612f-4e77-ba58-f16c8079169b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:35.818020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0407a92-612f-4e77-ba58-f16c8079169b admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:35.828113 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1464/2888] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:09:35.832284 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbf7ad-1905-4b17-a871-63cd1c85c244 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:35.843283 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbf7ad-1905-4b17-a871-63cd1c85c244 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:35.847886 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa6ad1-a545-488c-9ed8-61dabdfd5c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:35.848530 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa6ad1-a545-488c-9ed8-61dabdfd5c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:35.848812 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa6ad1-a545-488c-9ed8-61dabdfd5c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:35.851422 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa6ad1-a545-488c-9ed8-61dabdfd5c73 admin admin] PATH_INFO: `/v3/projects/b23d26bf57cc4c089506d5100840712e/users/598197f58e1d4505a3d1769652e107fc/roles/a2103068c87a47748ba790b7ea0dd26c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:35.853769 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa6ad1-a545-488c-9ed8-61dabdfd5c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23d26bf57cc4c089506d5100840712e, user_id=598197f58e1d4505a3d1769652e107fc, role_id=a2103068c87a47748ba790b7ea0dd26c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:35.857502 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa6ad1-a545-488c-9ed8-61dabdfd5c73 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:35.869468 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1425/2889] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:09:35 2022] PUT /identity/v3/projects/b23d26bf57cc4c089506d5100840712e/users/598197f58e1d4505a3d1769652e107fc/roles/a2103068c87a47748ba790b7ea0dd26c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:35.881378 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689076e-0c14-4d86-9b58-4e38d8ac858b None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:35.881718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689076e-0c14-4d86-9b58-4e38d8ac858b None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:35.882020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689076e-0c14-4d86-9b58-4e38d8ac858b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:35.917969 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-b689076e-0c14-4d86-9b58-4e38d8ac858b None None] MFA Rules not processed for user `598197f58e1d4505a3d1769652e107fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:35.918283 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b689076e-0c14-4d86-9b58-4e38d8ac858b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:36.005111 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1465/2890] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:36.018475 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa6ad1-a545-488c-9ed8-61dabdfd5c73 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:36.021168 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-1faa6ad1-a545-488c-9ed8-61dabdfd5c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:36.033065 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:36.089731 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa6ad1-a545-488c-9ed8-61dabdfd5c73 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:36.094342 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1154c69-db0c-4079-8061-364931aa8858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:36.095019 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1154c69-db0c-4079-8061-364931aa8858 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:36.095298 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1154c69-db0c-4079-8061-364931aa8858 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:36.095573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1154c69-db0c-4079-8061-364931aa8858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:36.166183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1154c69-db0c-4079-8061-364931aa8858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:36.167646 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1154c69-db0c-4079-8061-364931aa8858 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:36.177796 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1426/2891] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:38.111333 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689076e-0c14-4d86-9b58-4e38d8ac858b None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:38.112721 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b689076e-0c14-4d86-9b58-4e38d8ac858b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:38.119532 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:38.125346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689076e-0c14-4d86-9b58-4e38d8ac858b None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:38.128852 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e6007-5e30-42c4-9f4e-0c27ebf29378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:38.129476 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e6007-5e30-42c4-9f4e-0c27ebf29378 service cinder] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:38.129779 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e6007-5e30-42c4-9f4e-0c27ebf29378 service cinder] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:38.130048 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e6007-5e30-42c4-9f4e-0c27ebf29378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:38.187881 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e6007-5e30-42c4-9f4e-0c27ebf29378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:38.189338 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e6007-5e30-42c4-9f4e-0c27ebf29378 service cinder] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:38.198193 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1466/2892] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:39.699744 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1154c69-db0c-4079-8061-364931aa8858 service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:39.709320 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1154c69-db0c-4079-8061-364931aa8858 service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:39.713321 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b28973-f707-4722-8bd1-e4be5b12f2ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:39.713926 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b28973-f707-4722-8bd1-e4be5b12f2ca service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:39.714208 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b28973-f707-4722-8bd1-e4be5b12f2ca service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:39.714512 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b28973-f707-4722-8bd1-e4be5b12f2ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:39.724258 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b28973-f707-4722-8bd1-e4be5b12f2ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:39.731071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b28973-f707-4722-8bd1-e4be5b12f2ca service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:39.735294 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1427/2893] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:44.217922 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e6007-5e30-42c4-9f4e-0c27ebf29378 service cinder] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:44.278880 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e6007-5e30-42c4-9f4e-0c27ebf29378 service cinder] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:44.283672 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:44.284349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:44.284585 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:44.284856 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] PATH_INFO: `/v3/users/fcdc0ed4c88543dfb12c41c130aac500` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:44.285991 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcdc0ed4c88543dfb12c41c130aac500)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:44.287920 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:44.459043 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcdc0ed4c88543dfb12c41c130aac500'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:44.540660 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcdc0ed4c88543dfb12c41c130aac500'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:44.541287 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcdc0ed4c88543dfb12c41c130aac500'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:44.541905 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcdc0ed4c88543dfb12c41c130aac500'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:44.554182 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcdc0ed4c88543dfb12c41c130aac500'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:44.568348 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1467/2894] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:09:44 2022] DELETE /identity/v3/users/fcdc0ed4c88543dfb12c41c130aac500 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:44.572991 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66c47a-459d-4348-a5fe-ac47291beee5 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:44.573479 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66c47a-459d-4348-a5fe-ac47291beee5 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:44.573594 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66c47a-459d-4348-a5fe-ac47291beee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:44.593390 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-2d66c47a-459d-4348-a5fe-ac47291beee5 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:44.594615 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2d66c47a-459d-4348-a5fe-ac47291beee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:44.653000 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1428/2895] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:44.662844 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:44.664230 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:44.686766 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:44.721783 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90564086-a079-4fac-ba10-21cd016a2e77 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:44.725334 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92599169-2431-4d54-aa40-374a456ba320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:44.725930 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92599169-2431-4d54-aa40-374a456ba320 service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:44.726205 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92599169-2431-4d54-aa40-374a456ba320 service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:44.726474 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92599169-2431-4d54-aa40-374a456ba320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:44.736832 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92599169-2431-4d54-aa40-374a456ba320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:44.738267 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92599169-2431-4d54-aa40-374a456ba320 service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:44.747872 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1468/2896] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:45.145183 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66c47a-459d-4348-a5fe-ac47291beee5 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:45.146624 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-2d66c47a-459d-4348-a5fe-ac47291beee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:45.161362 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66c47a-459d-4348-a5fe-ac47291beee5 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:45.171661 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:45.172311 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:45.172593 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:45.172866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] PATH_INFO: `/v3/projects/7de0763651344f19911cfd82baa23add` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:45.174037 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7de0763651344f19911cfd82baa23add)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:45.176096 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:45.271705 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:45.275312 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7de0763651344f19911cfd82baa23add', 'request_id': 'req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:45.281385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7de0763651344f19911cfd82baa23add', 'request_id': 'req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:45.365163 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7de0763651344f19911cfd82baa23add', 'request_id': 'req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:45.376752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] The token cache is being invalidate because project 7de0763651344f19911cfd82baa23add was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:09:45.377249 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:45.378100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:45.379882 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1429/2897] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:09:45 2022] DELETE /identity/v3/projects/7de0763651344f19911cfd82baa23add => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:09:45.383988 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92599169-2431-4d54-aa40-374a456ba320 service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:45.394514 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92599169-2431-4d54-aa40-374a456ba320 service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:45.451229 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:45.451851 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:45.452122 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:45.452387 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] PATH_INFO: `/v3/users/bc25f538b4384ec583c4c9c71283af59` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:45.453475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc25f538b4384ec583c4c9c71283af59)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:45.455412 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:45.630630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc25f538b4384ec583c4c9c71283af59'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:45.713423 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc25f538b4384ec583c4c9c71283af59'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:45.714039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc25f538b4384ec583c4c9c71283af59'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:45.714736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc25f538b4384ec583c4c9c71283af59'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:45.727010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc25f538b4384ec583c4c9c71283af59'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:45.739817 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1469/2898] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:09:45 2022] DELETE /identity/v3/users/bc25f538b4384ec583c4c9c71283af59 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:46.138114 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:46.139643 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:46.155539 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a669f-5e56-43a0-a5d4-7b49f5b45e5d admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:46.213655 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:46.214280 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:46.214591 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:46.214866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] PATH_INFO: `/v3/projects/5d270690524b43269e0120cb3561abf0` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:46.216561 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d270690524b43269e0120cb3561abf0)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:46.218389 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:46.349207 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d270690524b43269e0120cb3561abf0', 'request_id': 'req-4e0c2864-47f9-40b2-994b-68100c2ca7f4'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:46.355341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d270690524b43269e0120cb3561abf0', 'request_id': 'req-4e0c2864-47f9-40b2-994b-68100c2ca7f4'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:46.428234 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d270690524b43269e0120cb3561abf0', 'request_id': 'req-4e0c2864-47f9-40b2-994b-68100c2ca7f4'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:46.439399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] The token cache is being invalidate because project 5d270690524b43269e0120cb3561abf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:09:46.439689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:46.443793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:46.443793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1430/2899] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:09:46 2022] DELETE /identity/v3/projects/5d270690524b43269e0120cb3561abf0 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:46.446272 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:46.447736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:46.461435 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d182f7-9087-481c-ae39-6c5a72c00d28 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:46.502126 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:46.518410 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293d187-0c19-4445-a64d-9432ca776f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:46.519065 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293d187-0c19-4445-a64d-9432ca776f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:46.519343 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293d187-0c19-4445-a64d-9432ca776f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:46.519638 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293d187-0c19-4445-a64d-9432ca776f38 admin admin] PATH_INFO: `/v3/users/598197f58e1d4505a3d1769652e107fc` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:46.520747 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293d187-0c19-4445-a64d-9432ca776f38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=598197f58e1d4505a3d1769652e107fc)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:46.522714 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293d187-0c19-4445-a64d-9432ca776f38 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:46.704039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f293d187-0c19-4445-a64d-9432ca776f38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '598197f58e1d4505a3d1769652e107fc'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:46.769516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f293d187-0c19-4445-a64d-9432ca776f38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '598197f58e1d4505a3d1769652e107fc'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:46.770124 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f293d187-0c19-4445-a64d-9432ca776f38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '598197f58e1d4505a3d1769652e107fc'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:46.770772 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f293d187-0c19-4445-a64d-9432ca776f38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '598197f58e1d4505a3d1769652e107fc'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:46.782810 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-f293d187-0c19-4445-a64d-9432ca776f38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '598197f58e1d4505a3d1769652e107fc'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:46.795062 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1470/2900] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:09:46 2022] DELETE /identity/v3/users/598197f58e1d4505a3d1769652e107fc => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:47.222376 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:47.230693 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:47.238897 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c2864-47f9-40b2-994b-68100c2ca7f4 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:47.295867 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:47.296497 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:47.296777 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:47.297050 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d admin admin] PATH_INFO: `/v3/projects/b23d26bf57cc4c089506d5100840712e` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:47.298156 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b23d26bf57cc4c089506d5100840712e)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:47.300087 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:47.302262 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:47.440430 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b23d26bf57cc4c089506d5100840712e', 'request_id': 'req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:47.446584 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b23d26bf57cc4c089506d5100840712e', 'request_id': 'req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:47.530932 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b23d26bf57cc4c089506d5100840712e', 'request_id': 'req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:47.541500 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d admin admin] The token cache is being invalidate because project b23d26bf57cc4c089506d5100840712e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:09:47.545761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:47.545761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-8f3eae61-9f3d-4e4d-be09-ee8eb3a06b3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:09:47.545761 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1431/2901] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:09:47 2022] DELETE /identity/v3/projects/b23d26bf57cc4c089506d5100840712e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:47.549607 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178da4c-1675-4276-9a5a-e3e842a82f0f None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:47.550095 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178da4c-1675-4276-9a5a-e3e842a82f0f None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:47.550218 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178da4c-1675-4276-9a5a-e3e842a82f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:47.569794 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-d178da4c-1675-4276-9a5a-e3e842a82f0f None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:47.571059 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-d178da4c-1675-4276-9a5a-e3e842a82f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:47.630617 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1471/2902] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:47.639818 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc774f-d4c1-4548-9b52-bf7991606e90 None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:47.640159 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc774f-d4c1-4548-9b52-bf7991606e90 None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:47.640440 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc774f-d4c1-4548-9b52-bf7991606e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:47.661399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-b6cc774f-d4c1-4548-9b52-bf7991606e90 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:47.662618 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-b6cc774f-d4c1-4548-9b52-bf7991606e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:47.671656 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1432/2903] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:47.676010 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178da4c-1675-4276-9a5a-e3e842a82f0f None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:47.686012 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178da4c-1675-4276-9a5a-e3e842a82f0f None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:47.690608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e60b47-0f73-4210-b45b-fc8d65f8b8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:47.691195 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e60b47-0f73-4210-b45b-fc8d65f8b8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:47.691469 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e60b47-0f73-4210-b45b-fc8d65f8b8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:47.691736 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e60b47-0f73-4210-b45b-fc8d65f8b8e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:47.692373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e60b47-0f73-4210-b45b-fc8d65f8b8e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:09:47.692668 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e60b47-0f73-4210-b45b-fc8d65f8b8e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:47.694301 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e60b47-0f73-4210-b45b-fc8d65f8b8e4 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:47.705946 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1472/2904] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:09:47.710221 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc774f-d4c1-4548-9b52-bf7991606e90 None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:47.720456 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc774f-d4c1-4548-9b52-bf7991606e90 None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:47.725097 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa133cae-7746-465b-bbe8-389bbfb05689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:47.725683 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa133cae-7746-465b-bbe8-389bbfb05689 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:47.725963 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa133cae-7746-465b-bbe8-389bbfb05689 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:47.726236 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa133cae-7746-465b-bbe8-389bbfb05689 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:47.726901 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa133cae-7746-465b-bbe8-389bbfb05689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:47.728830 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa133cae-7746-465b-bbe8-389bbfb05689 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:47.823791 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1433/2905] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:09:47 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:09:47.836366 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e60b47-0f73-4210-b45b-fc8d65f8b8e4 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:47.847020 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e60b47-0f73-4210-b45b-fc8d65f8b8e4 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:47.889633 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:47.901018 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceade79-91c2-490c-8090-78426ce1d6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:47.903853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceade79-91c2-490c-8090-78426ce1d6cd admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:47.903853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceade79-91c2-490c-8090-78426ce1d6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:47.903853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceade79-91c2-490c-8090-78426ce1d6cd admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:47.903853 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceade79-91c2-490c-8090-78426ce1d6cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:47.905806 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceade79-91c2-490c-8090-78426ce1d6cd admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:47.965320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1473/2906] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:09:47 2022] POST /identity/v3/users => generated 286 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:09:47.970063 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa133cae-7746-465b-bbe8-389bbfb05689 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:47.980324 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa133cae-7746-465b-bbe8-389bbfb05689 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:47.984992 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416ce0d-5197-4767-b995-29e0d152a944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:47.985588 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416ce0d-5197-4767-b995-29e0d152a944 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:47.985869 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416ce0d-5197-4767-b995-29e0d152a944 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:47.986141 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416ce0d-5197-4767-b995-29e0d152a944 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:47.986817 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416ce0d-5197-4767-b995-29e0d152a944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:47.988387 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416ce0d-5197-4767-b995-29e0d152a944 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:47.998325 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1434/2907] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:09:48.006778 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceade79-91c2-490c-8090-78426ce1d6cd admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:48.017442 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceade79-91c2-490c-8090-78426ce1d6cd admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:48.022337 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ca559-ef45-4bae-8e68-fba6b29d2fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:48.023241 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ca559-ef45-4bae-8e68-fba6b29d2fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:48.023519 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ca559-ef45-4bae-8e68-fba6b29d2fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:48.023792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ca559-ef45-4bae-8e68-fba6b29d2fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:48.024433 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ca559-ef45-4bae-8e68-fba6b29d2fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:48.025985 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ca559-ef45-4bae-8e68-fba6b29d2fe9 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:48.037492 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1474/2908] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:09:48.041576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416ce0d-5197-4767-b995-29e0d152a944 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:48.051563 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416ce0d-5197-4767-b995-29e0d152a944 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:48.056099 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9293da-ef3e-4ade-baa9-244a60a97fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:48.056727 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9293da-ef3e-4ade-baa9-244a60a97fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:48.057006 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9293da-ef3e-4ade-baa9-244a60a97fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:48.057275 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9293da-ef3e-4ade-baa9-244a60a97fc8 admin admin] PATH_INFO: `/v3/projects/813d39a2302342b18a9316581fa3a14e/users/89a1e89f834a495eb4f061eee7fd1676/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:48.071424 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9293da-ef3e-4ade-baa9-244a60a97fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=813d39a2302342b18a9316581fa3a14e, user_id=89a1e89f834a495eb4f061eee7fd1676, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:48.074076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9293da-ef3e-4ade-baa9-244a60a97fc8 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:48.154128 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1435/2909] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:09:48 2022] PUT /identity/v3/projects/813d39a2302342b18a9316581fa3a14e/users/89a1e89f834a495eb4f061eee7fd1676/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:09:48.159186 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd13155-4c78-4665-9166-0482b1fe2fb7 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:48.159472 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd13155-4c78-4665-9166-0482b1fe2fb7 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:48.159742 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd13155-4c78-4665-9166-0482b1fe2fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:48.178784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-7fd13155-4c78-4665-9166-0482b1fe2fb7 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:48.179976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7fd13155-4c78-4665-9166-0482b1fe2fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:48.240475 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1475/2910] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:48.250842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9293da-ef3e-4ade-baa9-244a60a97fc8 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:48.252573 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-8e9293da-ef3e-4ade-baa9-244a60a97fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:48.312142 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9293da-ef3e-4ade-baa9-244a60a97fc8 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:48.315783 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c3b10-a1f0-4df6-8a96-52ecc5e0f026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:48.316385 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c3b10-a1f0-4df6-8a96-52ecc5e0f026 service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:48.316660 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c3b10-a1f0-4df6-8a96-52ecc5e0f026 service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:48.316930 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c3b10-a1f0-4df6-8a96-52ecc5e0f026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:48.326788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c3b10-a1f0-4df6-8a96-52ecc5e0f026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:48.328173 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c3b10-a1f0-4df6-8a96-52ecc5e0f026 service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:48.337798 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1436/2911] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:49.572372 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dd8d1-01e4-4c29-b3ed-6f22e914c5d0 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:49.575976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dd8d1-01e4-4c29-b3ed-6f22e914c5d0 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:49.575976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dd8d1-01e4-4c29-b3ed-6f22e914c5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:49.604782 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-233dd8d1-01e4-4c29-b3ed-6f22e914c5d0 None None] MFA Rules not processed for user `89a1e89f834a495eb4f061eee7fd1676`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:49.606718 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-233dd8d1-01e4-4c29-b3ed-6f22e914c5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:49.677146 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1476/2912] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:49.687449 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c3b10-a1f0-4df6-8a96-52ecc5e0f026 service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:49.688890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5e4c3b10-a1f0-4df6-8a96-52ecc5e0f026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:49.723842 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:49.747332 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c3b10-a1f0-4df6-8a96-52ecc5e0f026 service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:49.750953 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5a4d8-5e76-4ef2-9a7f-a8c1d8e4d72f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:49.752441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5a4d8-5e76-4ef2-9a7f-a8c1d8e4d72f service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:49.752441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5a4d8-5e76-4ef2-9a7f-a8c1d8e4d72f service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:49.752441 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5a4d8-5e76-4ef2-9a7f-a8c1d8e4d72f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:49.760434 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5a4d8-5e76-4ef2-9a7f-a8c1d8e4d72f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:49.761858 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5a4d8-5e76-4ef2-9a7f-a8c1d8e4d72f service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:49.770974 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1437/2913] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:51.382705 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a7054-a990-4d41-99fd-2527a2c01d91 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:51.383223 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a7054-a990-4d41-99fd-2527a2c01d91 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:51.383349 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a7054-a990-4d41-99fd-2527a2c01d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:51.402606 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-de0a7054-a990-4d41-99fd-2527a2c01d91 None None] MFA Rules not processed for user `89a1e89f834a495eb4f061eee7fd1676`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:09:51.403800 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-de0a7054-a990-4d41-99fd-2527a2c01d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:51.410586 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1477/2914] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:09:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:09:51.652579 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5a4d8-5e76-4ef2-9a7f-a8c1d8e4d72f service cinder] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:51.662614 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5a4d8-5e76-4ef2-9a7f-a8c1d8e4d72f service cinder] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:51.666143 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235506b-58de-4580-aa5b-1747613bea0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:51.666803 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235506b-58de-4580-aa5b-1747613bea0f service neutron] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:51.667080 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235506b-58de-4580-aa5b-1747613bea0f service neutron] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:51.667347 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235506b-58de-4580-aa5b-1747613bea0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:51.669070 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a235506b-58de-4580-aa5b-1747613bea0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:51.678799 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235506b-58de-4580-aa5b-1747613bea0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:51.680244 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235506b-58de-4580-aa5b-1747613bea0f service neutron] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:51.688585 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1438/2915] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:09:52.081036 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a7054-a990-4d41-99fd-2527a2c01d91 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:52.082419 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-de0a7054-a990-4d41-99fd-2527a2c01d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:52.140974 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:09:52.158942 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a7054-a990-4d41-99fd-2527a2c01d91 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:52.163941 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a99f5-62a4-41b6-a6b8-82f5562d8b8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:52.164522 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a99f5-62a4-41b6-a6b8-82f5562d8b8e service placement] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:52.164799 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a99f5-62a4-41b6-a6b8-82f5562d8b8e service placement] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:52.165074 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a99f5-62a4-41b6-a6b8-82f5562d8b8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:52.166837 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-044a99f5-62a4-41b6-a6b8-82f5562d8b8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:52.178319 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a99f5-62a4-41b6-a6b8-82f5562d8b8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:52.179895 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a99f5-62a4-41b6-a6b8-82f5562d8b8e service placement] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:52.190593 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1478/2916] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:09:53.613279 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235506b-58de-4580-aa5b-1747613bea0f service neutron] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:09:53.614689 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-a235506b-58de-4580-aa5b-1747613bea0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:53.629329 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235506b-58de-4580-aa5b-1747613bea0f service neutron] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:09:53.634196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac95252-3a45-48d8-9eb3-c52dda3ed89b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:09:53.634826 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95252-3a45-48d8-9eb3-c52dda3ed89b service placement] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:09:53.635100 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95252-3a45-48d8-9eb3-c52dda3ed89b service placement] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:09:53.635367 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95252-3a45-48d8-9eb3-c52dda3ed89b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:09:53.637002 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-5ac95252-3a45-48d8-9eb3-c52dda3ed89b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:09:53.648082 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac95252-3a45-48d8-9eb3-c52dda3ed89b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:09:53.649620 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac95252-3a45-48d8-9eb3-c52dda3ed89b service placement] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:09:53.660202 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1439/2917] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:09:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:10:01.783458 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a99f5-62a4-41b6-a6b8-82f5562d8b8e service placement] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:10:01.784913 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-044a99f5-62a4-41b6-a6b8-82f5562d8b8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:10:01.860598 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a99f5-62a4-41b6-a6b8-82f5562d8b8e service placement] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:10:01.866071 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d86df1-fb64-4922-bf4f-9b99d3592306 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f29af98c5fba4569b16edf240b13c5c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:10:01.866706 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d86df1-fb64-4922-bf4f-9b99d3592306 service nova] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:01.866986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d86df1-fb64-4922-bf4f-9b99d3592306 service nova] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:01.867258 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d86df1-fb64-4922-bf4f-9b99d3592306 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:01.869000 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-b8d86df1-fb64-4922-bf4f-9b99d3592306 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:10:01.882277 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d86df1-fb64-4922-bf4f-9b99d3592306 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:10:01.883878 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d86df1-fb64-4922-bf4f-9b99d3592306 service nova] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:10:01.895393 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1479/2918] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:10:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 13:10:03.655832 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=24 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:10:08.534176 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8cece-719b-42a6-8ca8-6683042ee3be None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:08.534793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8cece-719b-42a6-8ca8-6683042ee3be None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:08.534793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8cece-719b-42a6-8ca8-6683042ee3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:08.554355 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-c6c8cece-719b-42a6-8ca8-6683042ee3be None None] MFA Rules not processed for user `89a1e89f834a495eb4f061eee7fd1676`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:10:08.555574 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c6c8cece-719b-42a6-8ca8-6683042ee3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:10:08.562752 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1440/2919] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:10:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:10:11.891516 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=43 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:10:18.548890 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:10:30.455909 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde0324-27df-429a-9f92-e05a54d3b97c None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:30.456420 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde0324-27df-429a-9f92-e05a54d3b97c None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:30.456547 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde0324-27df-429a-9f92-e05a54d3b97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:30.509369 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-1bde0324-27df-429a-9f92-e05a54d3b97c None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:10:30.510617 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-1bde0324-27df-429a-9f92-e05a54d3b97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:10:30.575013 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1480/2920] 10.222.0.46 () {58 vars in 1009 bytes} [Fri Sep 9 13:10:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:10:30.705226 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8cece-719b-42a6-8ca8-6683042ee3be None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:10:30.706965 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-c6c8cece-719b-42a6-8ca8-6683042ee3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:10:30.799509 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8cece-719b-42a6-8ca8-6683042ee3be None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:10:30.804558 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25b471-d288-49df-8822-29f042db76bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:10:30.805157 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25b471-d288-49df-8822-29f042db76bc service swift] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:30.805435 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25b471-d288-49df-8822-29f042db76bc service swift] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:30.805704 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25b471-d288-49df-8822-29f042db76bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:30.814182 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25b471-d288-49df-8822-29f042db76bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:10:30.815788 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25b471-d288-49df-8822-29f042db76bc service swift] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:10:30.824362 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1441/2921] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 13:10:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 13:10:40.572090 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=40 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:10:40.820875 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=41 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:10:47.366587 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf8be0-252b-494e-8cc1-d82ab37c05e2 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:47.367099 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf8be0-252b-494e-8cc1-d82ab37c05e2 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:47.367217 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf8be0-252b-494e-8cc1-d82ab37c05e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:47.388792 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-45cf8be0-252b-494e-8cc1-d82ab37c05e2 None None] MFA Rules not processed for user `ec77f4e9939b4dd685b850f8b6fb1e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:10:47.389910 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-45cf8be0-252b-494e-8cc1-d82ab37c05e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:10:47.397445 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1481/2922] 10.222.0.46 () {58 vars in 967 bytes} [Fri Sep 9 13:10:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:10:47.410465 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25b471-d288-49df-8822-29f042db76bc service swift] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:10:47.420818 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25b471-d288-49df-8822-29f042db76bc service swift] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:10:47.426678 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071032c-7417-491f-8d7d-f47b806e1737 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8e4805f0b1e484c98fb399313f88ef2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:10:47.427273 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071032c-7417-491f-8d7d-f47b806e1737 service swift] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:47.427552 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071032c-7417-491f-8d7d-f47b806e1737 service swift] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:47.427821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071032c-7417-491f-8d7d-f47b806e1737 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:47.435823 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071032c-7417-491f-8d7d-f47b806e1737 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:10:47.443144 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071032c-7417-491f-8d7d-f47b806e1737 service swift] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:10:47.445881 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1442/2923] 10.222.0.46 () {60 vars in 1384 bytes} [Fri Sep 9 13:10:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 13:10:50.330955 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf8be0-252b-494e-8cc1-d82ab37c05e2 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:10:50.342895 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf8be0-252b-494e-8cc1-d82ab37c05e2 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:10:50.347813 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d643eb-dfdb-486b-977f-54ca1584eb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:10:50.348530 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d643eb-dfdb-486b-977f-54ca1584eb8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:50.348864 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d643eb-dfdb-486b-977f-54ca1584eb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:50.349180 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d643eb-dfdb-486b-977f-54ca1584eb8d admin admin] PATH_INFO: `/v3/users/89a1e89f834a495eb4f061eee7fd1676` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:50.350294 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d643eb-dfdb-486b-977f-54ca1584eb8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=89a1e89f834a495eb4f061eee7fd1676)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:10:50.358293 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d643eb-dfdb-486b-977f-54ca1584eb8d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:10:50.554430 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a1d643eb-dfdb-486b-977f-54ca1584eb8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89a1e89f834a495eb4f061eee7fd1676'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:10:50.632182 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a1d643eb-dfdb-486b-977f-54ca1584eb8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89a1e89f834a495eb4f061eee7fd1676'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:10:50.632754 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a1d643eb-dfdb-486b-977f-54ca1584eb8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89a1e89f834a495eb4f061eee7fd1676'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:10:50.633376 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a1d643eb-dfdb-486b-977f-54ca1584eb8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89a1e89f834a495eb4f061eee7fd1676'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:10:50.646538 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-a1d643eb-dfdb-486b-977f-54ca1584eb8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89a1e89f834a495eb4f061eee7fd1676'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:10:50.660771 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1482/2924] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:10:50 2022] DELETE /identity/v3/users/89a1e89f834a495eb4f061eee7fd1676 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:10:50.943043 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071032c-7417-491f-8d7d-f47b806e1737 service swift] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:10:50.956619 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071032c-7417-491f-8d7d-f47b806e1737 service swift] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:10:51.035296 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996821d-3b34-4cac-a987-2c4224ca9186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:10:51.036098 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996821d-3b34-4cac-a987-2c4224ca9186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:51.036418 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996821d-3b34-4cac-a987-2c4224ca9186 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:51.036762 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996821d-3b34-4cac-a987-2c4224ca9186 admin admin] PATH_INFO: `/v3/projects/813d39a2302342b18a9316581fa3a14e` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:51.037949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996821d-3b34-4cac-a987-2c4224ca9186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=813d39a2302342b18a9316581fa3a14e)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:10:51.039907 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996821d-3b34-4cac-a987-2c4224ca9186 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:10:51.152366 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e996821d-3b34-4cac-a987-2c4224ca9186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '813d39a2302342b18a9316581fa3a14e', 'request_id': 'req-e996821d-3b34-4cac-a987-2c4224ca9186'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:10:51.159267 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e996821d-3b34-4cac-a987-2c4224ca9186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '813d39a2302342b18a9316581fa3a14e', 'request_id': 'req-e996821d-3b34-4cac-a987-2c4224ca9186'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:10:51.243542 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e996821d-3b34-4cac-a987-2c4224ca9186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '813d39a2302342b18a9316581fa3a14e', 'request_id': 'req-e996821d-3b34-4cac-a987-2c4224ca9186'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:10:51.255450 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e996821d-3b34-4cac-a987-2c4224ca9186 admin admin] The token cache is being invalidate because project 813d39a2302342b18a9316581fa3a14e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:10:51.255735 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e996821d-3b34-4cac-a987-2c4224ca9186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:10:51.256462 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-e996821d-3b34-4cac-a987-2c4224ca9186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:10:51.257949 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1443/2925] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:10:50 2022] DELETE /identity/v3/projects/813d39a2302342b18a9316581fa3a14e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:10:51.262948 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60d897-ebce-4122-a2a9-6ca808d55a33 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:51.263234 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60d897-ebce-4122-a2a9-6ca808d55a33 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:51.263509 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60d897-ebce-4122-a2a9-6ca808d55a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:51.284590 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-0b60d897-ebce-4122-a2a9-6ca808d55a33 None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:10:51.285784 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-0b60d897-ebce-4122-a2a9-6ca808d55a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:10:51.345211 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1483/2926] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:10:51.352816 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a621a10-bfe1-48d3-8418-5a0926e73d2e None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:51.353102 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a621a10-bfe1-48d3-8418-5a0926e73d2e None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:51.353375 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a621a10-bfe1-48d3-8418-5a0926e73d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:51.373476 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-7a621a10-bfe1-48d3-8418-5a0926e73d2e None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:10:51.374706 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-7a621a10-bfe1-48d3-8418-5a0926e73d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:10:51.382836 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1444/2927] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:10:51.386971 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60d897-ebce-4122-a2a9-6ca808d55a33 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:10:51.397876 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60d897-ebce-4122-a2a9-6ca808d55a33 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:10:51.402355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89ce22-8cf3-47be-892d-ceb2e2654840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:10:51.402986 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89ce22-8cf3-47be-892d-ceb2e2654840 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:51.403262 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89ce22-8cf3-47be-892d-ceb2e2654840 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:51.403529 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89ce22-8cf3-47be-892d-ceb2e2654840 admin admin] PATH_INFO: `/v3/domains` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:51.404163 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89ce22-8cf3-47be-892d-ceb2e2654840 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88041) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 13:10:51.404457 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89ce22-8cf3-47be-892d-ceb2e2654840 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:10:51.406014 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89ce22-8cf3-47be-892d-ceb2e2654840 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:10:51.417725 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1484/2928] 10.222.0.46 () {58 vars in 1153 bytes} [Fri Sep 9 13:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 13:10:51.422455 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a621a10-bfe1-48d3-8418-5a0926e73d2e None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:10:51.431524 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a621a10-bfe1-48d3-8418-5a0926e73d2e None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:10:51.436071 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ff126-221f-4ab3-9c69-2be7ade119c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:10:51.436654 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ff126-221f-4ab3-9c69-2be7ade119c4 admin admin] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:51.436929 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ff126-221f-4ab3-9c69-2be7ade119c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:51.437196 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ff126-221f-4ab3-9c69-2be7ade119c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:51.438765 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ff126-221f-4ab3-9c69-2be7ade119c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:10:51.440677 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ff126-221f-4ab3-9c69-2be7ade119c4 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:10:51.524576 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1445/2929] 10.222.0.46 () {60 vars in 1153 bytes} [Fri Sep 9 13:10:51 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 13:10:51.528684 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89ce22-8cf3-47be-892d-ceb2e2654840 admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:10:51.538768 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89ce22-8cf3-47be-892d-ceb2e2654840 admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:10:51.585926 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:10:51.594630 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f681cd-d715-4eac-bab0-54f57c917f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:10:51.595228 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f681cd-d715-4eac-bab0-54f57c917f3d admin admin] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:51.596895 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f681cd-d715-4eac-bab0-54f57c917f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:51.596895 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f681cd-d715-4eac-bab0-54f57c917f3d admin admin] PATH_INFO: `/v3/users` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:51.596895 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f681cd-d715-4eac-bab0-54f57c917f3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:10:51.598228 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f681cd-d715-4eac-bab0-54f57c917f3d admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:10:51.699584 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1485/2930] 10.222.0.46 () {60 vars in 1144 bytes} [Fri Sep 9 13:10:51 2022] POST /identity/v3/users => generated 283 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 13:10:51.704407 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ff126-221f-4ab3-9c69-2be7ade119c4 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:10:51.715001 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ff126-221f-4ab3-9c69-2be7ade119c4 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:10:51.720866 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6a4e3-487b-4089-88c9-1ec5f1997981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:10:51.721487 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6a4e3-487b-4089-88c9-1ec5f1997981 admin admin] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:51.721793 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6a4e3-487b-4089-88c9-1ec5f1997981 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:51.722122 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6a4e3-487b-4089-88c9-1ec5f1997981 admin admin] PATH_INFO: `/v3/roles` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:51.722798 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6a4e3-487b-4089-88c9-1ec5f1997981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:10:51.724368 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6a4e3-487b-4089-88c9-1ec5f1997981 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:10:51.734076 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1446/2931] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:10:51.738657 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f681cd-d715-4eac-bab0-54f57c917f3d admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:10:51.748280 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f681cd-d715-4eac-bab0-54f57c917f3d admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:10:51.752874 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef749618-a455-4860-b604-c734b716ed61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:10:51.755373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef749618-a455-4860-b604-c734b716ed61 admin admin] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:51.755373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef749618-a455-4860-b604-c734b716ed61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:51.755373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef749618-a455-4860-b604-c734b716ed61 admin admin] PATH_INFO: `/v3/roles` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:51.755373 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef749618-a455-4860-b604-c734b716ed61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:10:51.756215 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef749618-a455-4860-b604-c734b716ed61 admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:10:51.766508 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1486/2932] 10.222.0.46 () {58 vars in 1122 bytes} [Fri Sep 9 13:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 13:10:51.770710 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6a4e3-487b-4089-88c9-1ec5f1997981 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:10:51.785024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6a4e3-487b-4089-88c9-1ec5f1997981 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:10:51.790248 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97d4e9-b746-4701-b658-6bd6e0332dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:10:51.790931 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97d4e9-b746-4701-b658-6bd6e0332dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:51.791211 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97d4e9-b746-4701-b658-6bd6e0332dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:51.791481 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97d4e9-b746-4701-b658-6bd6e0332dd0 admin admin] PATH_INFO: `/v3/projects/d5dee2c22f03423c834aef827d183269/users/f912c2e4dc76447a9e83dea29bdf9d69/roles/43667d074eda4143bf3b7f721e83810c` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:51.806230 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97d4e9-b746-4701-b658-6bd6e0332dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5dee2c22f03423c834aef827d183269, user_id=f912c2e4dc76447a9e83dea29bdf9d69, role_id=43667d074eda4143bf3b7f721e83810c)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:10:51.808864 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97d4e9-b746-4701-b658-6bd6e0332dd0 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:10:51.878204 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1447/2933] 10.222.0.46 () {60 vars in 1483 bytes} [Fri Sep 9 13:10:51 2022] PUT /identity/v3/projects/d5dee2c22f03423c834aef827d183269/users/f912c2e4dc76447a9e83dea29bdf9d69/roles/43667d074eda4143bf3b7f721e83810c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:10:51.888998 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c49c4-632f-433b-a728-cf058b468e67 None None] REQUEST_METHOD: `POST` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:51.889284 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c49c4-632f-433b-a728-cf058b468e67 None None] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:51.889560 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c49c4-632f-433b-a728-cf058b468e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:51.917214 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.auth.core [None req-e77c49c4-632f-433b-a728-cf058b468e67 None None] MFA Rules not processed for user `f912c2e4dc76447a9e83dea29bdf9d69`. Rule list: `[]` (Enabled: `True`). {{(pid=88041) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:10:51.918418 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-e77c49c4-632f-433b-a728-cf058b468e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:10:51.981168 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1487/2934] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:10:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:10:51.992917 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97d4e9-b746-4701-b658-6bd6e0332dd0 admin admin] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:10:51.994690 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-0d97d4e9-b746-4701-b658-6bd6e0332dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:10:52.009024 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:10:52.054191 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97d4e9-b746-4701-b658-6bd6e0332dd0 admin admin] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:10:52.058505 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeaa034-ac2a-4a8d-8962-c471505f5e50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '002c117a85c1452e9fae80c755c6622f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:10:52.059119 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaa034-ac2a-4a8d-8962-c471505f5e50 service cinder] REQUEST_METHOD: `GET` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:10:52.059401 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaa034-ac2a-4a8d-8962-c471505f5e50 service cinder] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:10:52.059674 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaa034-ac2a-4a8d-8962-c471505f5e50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:10:52.068205 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeaa034-ac2a-4a8d-8962-c471505f5e50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:10:52.069623 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeaa034-ac2a-4a8d-8962-c471505f5e50 service cinder] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:10:52.078658 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1448/2935] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:10:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:11:01.978354 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:02.079781 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=20 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:29.505824 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c49c4-632f-433b-a728-cf058b468e67 None None] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:11:29.568841 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c49c4-632f-433b-a728-cf058b468e67 None None] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:11:29.573321 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:11:29.573932 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:29.574208 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:29.574499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] PATH_INFO: `/v3/users/f912c2e4dc76447a9e83dea29bdf9d69` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:29.575601 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=f912c2e4dc76447a9e83dea29bdf9d69)` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:11:29.577524 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:11:29.753408 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f912c2e4dc76447a9e83dea29bdf9d69'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:29.808246 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f912c2e4dc76447a9e83dea29bdf9d69'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:29.808868 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f912c2e4dc76447a9e83dea29bdf9d69'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:29.809521 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f912c2e4dc76447a9e83dea29bdf9d69'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:29.822700 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.notifications [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f912c2e4dc76447a9e83dea29bdf9d69'} {{(pid=88041) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:29.835147 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1488/2936] 10.222.0.46 () {58 vars in 1224 bytes} [Fri Sep 9 13:11:29 2022] DELETE /identity/v3/users/f912c2e4dc76447a9e83dea29bdf9d69 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 13:11:29.839839 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4209e-2210-4317-8331-01fe94a5ec0b None None] REQUEST_METHOD: `POST` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:29.840358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4209e-2210-4317-8331-01fe94a5ec0b None None] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:29.840358 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4209e-2210-4317-8331-01fe94a5ec0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:29.858959 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.auth.core [None req-f6e4209e-2210-4317-8331-01fe94a5ec0b None None] MFA Rules not processed for user `9947df19bb3b44318e674b5168544c18`. Rule list: `[]` (Enabled: `True`). {{(pid=88042) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 13:11:29.860133 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f6e4209e-2210-4317-8331-01fe94a5ec0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:29.919195 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1449/2937] 10.222.0.46 () {58 vars in 958 bytes} [Fri Sep 9 13:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 13:11:29.929499 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:11:29.930952 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:29.987976 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e735ae6-66aa-468e-b8a1-c831ce4746ca admin admin] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:11:29.991770 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e498c28-385e-471f-9ace-1180ce304f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e46bd9d49bf64013ae58770b0284d81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:11:29.995292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e498c28-385e-471f-9ace-1180ce304f8e service neutron] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:29.995292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e498c28-385e-471f-9ace-1180ce304f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:29.995292 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e498c28-385e-471f-9ace-1180ce304f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:30.001669 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e498c28-385e-471f-9ace-1180ce304f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:11:30.003123 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e498c28-385e-471f-9ace-1180ce304f8e service neutron] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:11:30.012346 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1489/2938] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:11:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:11:30.387459 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4209e-2210-4317-8331-01fe94a5ec0b None None] Authenticating user token {{(pid=88042) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:11:30.388977 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.fernet_utils [None req-f6e4209e-2210-4317-8331-01fe94a5ec0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88042) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:11:30.401850 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4209e-2210-4317-8331-01fe94a5ec0b None None] Validating token access rules against request {{(pid=88042) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:11:30.410821 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff830b-ddff-4745-bd74-b98043e449d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9947df19bb3b44318e674b5168544c18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e314473e6b1442dfa9c0bcc6fc5f97c6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88042) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:11:30.411916 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff830b-ddff-4745-bd74-b98043e449d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:11:30.412442 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff830b-ddff-4745-bd74-b98043e449d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:11:30.412983 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff830b-ddff-4745-bd74-b98043e449d7 admin admin] PATH_INFO: `/v3/projects/d5dee2c22f03423c834aef827d183269` {{(pid=88042) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:11:30.414964 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff830b-ddff-4745-bd74-b98043e449d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5dee2c22f03423c834aef827d183269)` {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:11:30.417787 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff830b-ddff-4745-bd74-b98043e449d7 admin admin] RBAC: Authorization granted {{(pid=88042) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:11:30.499839 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-24ff830b-ddff-4745-bd74-b98043e449d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5dee2c22f03423c834aef827d183269', 'request_id': 'req-24ff830b-ddff-4745-bd74-b98043e449d7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:30.505900 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-24ff830b-ddff-4745-bd74-b98043e449d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5dee2c22f03423c834aef827d183269', 'request_id': 'req-24ff830b-ddff-4745-bd74-b98043e449d7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:30.562707 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-24ff830b-ddff-4745-bd74-b98043e449d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5dee2c22f03423c834aef827d183269', 'request_id': 'req-24ff830b-ddff-4745-bd74-b98043e449d7'} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:30.573399 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-24ff830b-ddff-4745-bd74-b98043e449d7 admin admin] The token cache is being invalidate because project d5dee2c22f03423c834aef827d183269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88042) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 13:11:30.573683 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-24ff830b-ddff-4745-bd74-b98043e449d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:30.574302 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG keystone.notifications [None req-24ff830b-ddff-4745-bd74-b98043e449d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88042) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 13:11:30.575802 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: [pid: 88042|app: 0|req: 1450/2939] 10.222.0.46 () {58 vars in 1233 bytes} [Fri Sep 9 13:11:30 2022] DELETE /identity/v3/projects/d5dee2c22f03423c834aef827d183269 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 13:11:40.008189 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:11:40.570341 dv-cinder-813229-14-iscsi devstack@keystone.service[88042]: DEBUG dbcounter [-] [88042] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88042) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 13:15:46.299050 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e498c28-385e-471f-9ace-1180ce304f8e service neutron] Authenticating user token {{(pid=88041) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 13:15:46.300594 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-7e498c28-385e-471f-9ace-1180ce304f8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:46.445534 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e498c28-385e-471f-9ace-1180ce304f8e service neutron] Validating token access rules against request {{(pid=88041) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 13:15:46.503175 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c613af0-a5cd-4a56-8c65-53fc7c0e9479 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e177d86450489cacc86578ba26509d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e2dc716277d42ae8316ebb2d36293a8', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88041) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 13:15:46.503764 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613af0-a5cd-4a56-8c65-53fc7c0e9479 service placement] REQUEST_METHOD: `GET` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 13:15:46.504039 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613af0-a5cd-4a56-8c65-53fc7c0e9479 service placement] SCRIPT_NAME: `/identity` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 13:15:46.504307 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613af0-a5cd-4a56-8c65-53fc7c0e9479 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88041) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 13:15:46.505992 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.fernet_utils [None req-5c613af0-a5cd-4a56-8c65-53fc7c0e9479 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88041) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 13:15:46.517320 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c613af0-a5cd-4a56-8c65-53fc7c0e9479 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 13:15:46.518904 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c613af0-a5cd-4a56-8c65-53fc7c0e9479 service placement] RBAC: Authorization granted {{(pid=88041) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 13:15:46.529608 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: [pid: 88041|app: 0|req: 1490/2940] 10.222.0.46 () {60 vars in 1365 bytes} [Fri Sep 9 13:15:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 13:15:56.524355 dv-cinder-813229-14-iscsi devstack@keystone.service[88041]: DEBUG dbcounter [-] [88041] Writing DB stats keystone:SELECT=89 {{(pid=88041) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}